Skip to content

A parser that meets a field number its schema doesn't define doesn't just discard it. The wire format already says how many bytes the value takes up, so the parser reads them and holds onto them as an unknown field, then keeps going. Serialize that message again, and those bytes come right back out.

This round-tripping is the machinery behind Protobuf's compatibility guarantees. An old client can accept a message from a new server, touch the fields it knows about, and pass the message along without stripping whatever got added since it was compiled. The same goes for proxies and queues sitting between services that upgrade on different schedules.

The guarantee is narrower than it sounds, though. Unknown fields live on the decoded message instance. They only get written back out when that same instance, of that same message type, is serialized again, and plenty of services never do that. Unpack a request into a domain object, build a fresh response (or even a fresh copy of the same type, field by field), and you end up with a message that has no unknown fields at all. Some hops in a pipeline forward messages untouched. Others translate them into something new. Preservation stops at that second kind of hop.

Be careful with this at the edge of your system, though. A payload from outside can carry unknown fields your validation never inspected, and preservation will happily ferry them straight to internal services that do understand them. Plenty of teams that expose Protobuf APIs to the outside world drop unknown fields at the boundary as a precaution. Most runtimes give you a discard option when parsing.

USER.PROTO (WRITER, V2)
message User {
  string id = 1;
  string email = 2;
}
sending
{ "id": "u_1", "email": "a@b.c" }
serialize
ON THE WIRE
0a03755f31
field 1 — id
12056140622e63
field 2 — email
parsed by a reader compiled against v1
USER.PROTO (READER, V1)
message User {
  string id = 1;
}
parsed into a field
id = "u_1"
kept as an unknown field
12056140622e63
then the reader passes the message along
RE-SERIALIZES THE SAME INSTANCE
send(user)
0a03755f3112056140622e63
unknown bytes written back out
REBUILDS THE MESSAGE
send(toProto(toDomain(user)))
0a03755f3112 05 61 40 62 2e 63
field 2 — dropped: a new instance has no unknown fields

Unknown fields live on the decoded message instance. They survive only when that same instance, of that same message type, is serialized again.

Further Reading

Next

Schema Evolution

Which Protobuf schema changes are safe and which break deployed clients: wire and JSON compatibility levels, plus the deprecate-and-reserve lifecycle for removing fields.