Skip to content

PROTOBUF
EXPLAINED

Learn about important Protobuf concepts in a more interactive and visual way. Learn how Protobuf works, from the language to the binary encoding.

Protocol Buffers (Protobuf) is a schema-driven format for serializing structured data.

  • Shared Contract

    Write your type definitions once and use the same ones across every team and language that uses them.

  • Type Safety

    Generate types from the schema so your compiler catches wrong field names and wrong types before anything runs.

  • Compatibility

    Add fields and keep already-deployed clients working, as long as you follow Protobuf's compatibility rules. A client built against an older schema ignores what it does not recognize in the binary encoding. ProtoJSON readers reject unknown fields by default, so they need more care.

How it works

Protocol Buffers, or "Protobuf", refers to two things: an Interface Definition Language (IDL) for writing schemas, and the encodings those schemas produce. This guide covers the two encodings you will use in practice. One is a compact binary form; the other is JSON, which uses a standardized JSON mapping. Both encodings are part of the specification, and either can go over the wire. Code generated from a schema can read and write both representations.

A JSON payload carries field names and readable values as text, so you (or any JSON tool) can read it without the schema. The binary encoding leaves the names out and identifies fields only by number, so a decoder needs the schema to know that field 2 is name.

Protobuf filemessage User { int64 id = 1; string name = 2; bool verified = 3;}Dataid: 1042name: "Hiro"verified: true+Protobuf binary11 bytes08 92 08id12 04 48 69 72 6fname18 01verifiedJSON43 bytes{ "id": "1042", "name": "Hiro", "verified": true}

The Binary section explains how those bytes are laid out.

From schema to code

A compiler reads the schema and generates native code for each language you target: structs in Go, types in TypeScript, classes in Java, and so on. The generated code gives you typed constructors, the binary serialization, and the JSON mapping, so a message behaves like a native type in your language.

Next

Basics

Messages, fields, field numbers, enums, packages, and collections, then generating code from a schema and serving it over RPC.