bug with protobuf Any

Recently, I have implemented solution leveraging on protobuf.

The message are something like

message PositionGroup {
   repeated Position payloads = 1;
}
message Position {
  .....
}

There are several messages each with around 1 million repeated fields (the payloads). While the front tier (a graphql) return the combined result (expecting several million of records back), however, always return 1 million fields.

It took me quite a while to troubleshoot till surprisingly found out the protobuf implementation is totally different from what the API (mergeFrom) sounds like, make me almost feel like it’s a prank.

Per spec, “mergeFrom(Message other): (builder only) merges the contents of other into this message, overwriting singular scalar fields, merging composite fields, and concatenating repeated fields” so it does seem that the repeated payloads fields should be concatenated instead of replaced.

https://github.com/protocolbuffers/protobuf/issues/10021

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s