Skip to content

Schema

IMB uses Google's Protocol Buffers. To avoid using .proto files there is a Schema service which translates field names to keys (Protocol Buffers tag/wire-type). This makes the structures in IMB5 dynamic of type and name but the data files in the store have to match the schema and cannot be decoded without the correct version of the schema files. On the other hand data is more compact because, next to the value, only the key, an integer, for each field has to be stored.

  • Tag: all Protocol Buffers data is tagged. The tag is used to make streamed data backwards compatible; only fields with a known tag have to be processed.
  • Wire type: Protocol Buffers defines a wire type combined with a tag for each data element. When streaming data, fields with an unknown tag can be skipped. Due to the wire type stream processors know how to skip unknown fields i.e. fields that contain data not of interest to them.
  • Key: a key is the combination of a tag and wire-type. The two are combined following the Protocol Buffers protocol. The key is read as a single element and then is all that is needed to read the following data or skip that data in the stream.

The schema service connects the key in the data stream to field names that are used by models to identify properties of objects they use internally.