Extracting Protobuf custom option from file descriptor set?

Question

Suppose we have a file named foo.proto defining a message and a custom option:

syntax = "proto3";

package foo_package;

import "google/protobuf/descriptor.proto";

enum State {
    ALPHA = 0;
    BETA = 1;
}

extend google.protobuf.FieldOptions {
    State baz = 51234;
}

message Foo {
    string bar = 1 [ (baz) = ALPHA ];
}                                                                                                                                            

We generate a FileDescriptorSet (from a directory containing google/protobuf/descriptor.proto) for this message via:

protoc -I=. --include_imports -oTMP ./foo.proto

How can I extract baz from a message class instance in this set? The documentation and library (1), (2) suggest something like this might work:

from google.protobuf.descriptor_pb2 import FileDescriptorSet
from google.protobuf.message_factory import GetMessages

with open("TMP", mode="rb") as f:
    fds = FileDescriptorSet.FromString(f.read())

    messages = GetMessages([file for file in fds.file])
    extensions = messages["foo_package.Foo"].DESCRIPTOR.fields_by_name["bar"].GetOptions().Extensions

but the resulting object is empty. #6662 implies using a DescriptorPool should resolve it, but that also doesn't seem to work (empty object, as well).

Answer 1

GetOptions() will give you the built-in non-extended type google.protobuf.FieldOptions. In the protobuf implementation, GetOptions() constructs a message of the built-in type google.protobuf.FieldOptions, and then parses the bytes into that struct. You can see that here in the Python impl, for e.g.: https://github.com/protocolbuffers/protobuf/blob/5df4c2ec9426b06dfe8a019ddcf1509b8816cebe/python/google/protobuf/descriptor.py#L167-L170

The message descriptor will indeed have the options you defined; you can print the serialized version of the field options and you will see them, but they are not parsed into the resulting field options. The non-extended type does not describe those additional fields, so they're ignored.

What you actually want is to parse the options using your extended version of google.protobuf.FieldOptions.

I'm not sure if there's a more elegant way to do this, but what worked for me is to re-serialize the options, and then parse them using the dynamic version of your extended FieldOptions. Overall that looks something like this:

messages = GetMessages([file for file in fds.file])
Foo = messages["foo_package.Foo"]
FieldOptions = messages["google.protobuf.FieldOptions"]

bar_builtin_opts = Foo.DESCRIPTOR.fields_by_name["bar"].GetOptions()
bar_opts = FieldOptions()
bar_opts.ParseFromString(bar_builtin_opts.SerializeToString())

Note that this process is the same for all option types (field options, message options, etc).

You may have to create your own descriptor pool & message factory that is separate from the default ones if you run into issues redefining existing message types, though that's as simple as

pool = DescriptorPool()
for fd in fds.file:
    pool.Add(fd)
factory = MessageFactory(pool)
messages = factory.GetMessages([file.name for file in fds.file])