protobuf.com

Binary

Binary 101

Protobuf's encoding leans on three bit-level concepts. There's byte order, bit shifting, and bitwise OR. If you can already read x << 3 | y, skip ahead. If not, here's the short version.

Endianness

Multi-byte numbers can be laid out smallest byte first (little-endian) or largest byte first (big-endian). Protobuf's fixed-width types (fixed32, fixed64, float, double) are little-endian. That's why the low bytes of a number show up first in the raw stream.

00000011111010001110100000000011

Bit Shifting

A left shift (<<) moves every bit left and fills the gap with zeroes. Shift by 3 and you multiply by 8, leaving 3 empty bits open on the right. Protobuf uses this to build the field tag. It shifts the field number left by 3 to make room for the wire type.

0000000100000001000

Bitwise OR

OR (|) merges two numbers. An output bit is 1 if either input bit is 1. After a shift clears space, OR glues another value into it. As long as the two values occupy non-overlapping bit ranges, neither one corrupts the other.

000110000000101000011010

Further Reading

  • Endianness

    Byte order in full: where little- and big-endian come from, and why the distinction matters the moment bytes leave one machine for another.

  • Bit shifts

    The shift operations behind the tag formula, including the arithmetic/logical distinction this page glosses over.

  • Bitwise OR

    How OR merges bit patterns, alongside the AND and XOR siblings that decoders lean on.

Next

Base-128 Varints

How Protobuf base-128 varints encode integers in as few bytes as possible: continuation bits, byte layout, and an interactive varint encoder.

Previous
Binary
Next
Base-128 Varints