Python Generated Code Guide

Any differences between proto2, proto3, and Editions generated code are highlighted - note that these differences are in the generated code as described in this document, not the base message classes/interfaces, which are the same across all versions. You should read the proto2 language guide, proto3 language guide, and/or Editions guide before reading this document.

The Python Protocol Buffers implementation is a little different from C++ and Java. In Python, the compiler only outputs code to build descriptors for the generated classes, and a Python metaclass does the real work. This document describes what you get after the metaclass has been applied.

Compiler Invocation

The protocol buffer compiler produces Python output when invoked with the --python_out= command-line flag. The parameter to the --python_out= option is the directory where you want the compiler to write your Python output. The compiler creates a .py file for each .proto file input. The names of the output files are computed by taking the name of the .proto file and making two changes:

• The extension (.proto) is replaced with _pb2.py.
• The proto path (specified with the --proto_path= or -I command-line flag) is replaced with the output path (specified with the --python_out= flag).

So, for example, let’s say you invoke the compiler as follows:

protoc --proto_path=src --python_out=build/gen src/foo.proto src/bar/baz.proto

The compiler will read the files src/foo.proto and src/bar/baz.proto and produce two output files: build/gen/foo_pb2.py and build/gen/bar/baz_pb2.py. The compiler will automatically create the directory build/gen/bar if necessary, but it will not create build or build/gen; they must already exist.

Protoc can generate Python stubs (.pyi) using the --pyi_out parameter.

Note that if the .proto file or its path contains any characters which cannot be used in Python module names (for example, hyphens), they will be replaced with underscores. So, the file foo-bar.proto becomes the Python file foo_bar_pb2.py.

Packages

The Python code generated by the protocol buffer compiler is completely unaffected by the package name defined in the .proto file. Instead, Python packages are identified by directory structure.

Messages

Given a simple message declaration:

message Foo {}

The protocol buffer compiler generates a class called Foo, which subclasses google.protobuf.Message. The class is a concrete class; no abstract methods are left unimplemented. Unlike C++ and Java, Python generated code is unaffected by the optimize_for option in the .proto file; in effect, all Python code is optimized for code size.

If the message’s name is a Python keyword, then its class will only be accessible via getattr(), as described in the Names that conflict with Python keywords section.

You should not create your own Foo subclasses. Generated classes are not designed for subclassing and may lead to "fragile base class" problems. Besides, implementation inheritance is bad design.

Python message classes have no particular public members other than those defined by the Message interface and those generated for nested fields, messages, and enum types (described below). Message provides methods you can use to check, manipulate, read, or write the entire message, including parsing from and serializing to binary strings. In addition to these methods, the Foo class defines the following static methods:

• FromString(s): Returns a new message instance deserialized from the given string.

Note that you can also use the text_format module to work with protocol messages in text format: for example, the Merge() method lets you merge an ASCII representation of a message into an existing message.

Nested Types

A message can be declared inside another message. For example:

message Foo {
  message Bar {}
}

In this case, the Bar class is declared as a static member of Foo, so you can refer to it as Foo.Bar.

Well-known Types

Protocol buffers provides a number of well-known types that you can use in your .proto files along with your own message types. Some WKT messages have special methods in addition to the usual protocol buffer message methods, as they subclass both google.protobuf.Message and a WKT class.

Any

For Any messages, you can call Pack() to pack a specified message into the current Any message, or Unpack() to unpack the current Any message into a specified message. For example:

any_message.Pack(message)
any_message.Unpack(message)

Unpack() also checks the descriptor of the passed-in message object against the stored one and returns False if they don’t match and does not attempt any unpacking; True otherwise.

You can also call the Is() method to check if the Any message represents the given protocol buffer type. For example:

assert any_message.Is(message.DESCRIPTOR)

Use the TypeName() method to retrieve the protobuf type name of an inner message.

Timestamp

Timestamp messages can be converted to/from RFC 3339 date string format (JSON string) using the ToJsonString()/FromJsonString() methods. For example:

timestamp_message.FromJsonString("1970-01-01T00:00:00Z")
assert timestamp_message.ToJsonString() == "1970-01-01T00:00:00Z"

You can also call GetCurrentTime() to fill the Timestamp message with current time:

timestamp_message.GetCurrentTime()

To convert between other time units since epoch, you can call ToNanoseconds(), FromNanoseconds(), ToMicroseconds(), FromMicroseconds(), ToMilliseconds(), FromMilliseconds(), ToSeconds(), or FromSeconds(). The generated code also has ToDatetime() and FromDatetime() methods to convert between Python datetime objects and Timestamps. For example:

timestamp_message.FromMicroseconds(-1)
assert timestamp_message.ToMicroseconds() == -1
dt = datetime(2016, 1, 1)
timestamp_message.FromDatetime(dt)
self.assertEqual(dt, timestamp_message.ToDatetime())

Duration

Duration messages have the same methods as Timestamp to convert between JSON string and other time units. To convert between timedelta and Duration, you can call ToTimedelta() or FromTimedelta. For example:

duration_message.FromNanoseconds(1999999999)
td = duration_message.ToTimedelta()
assert td.seconds == 1
assert td.microseconds == 999999

FieldMask

FieldMask messages can be converted to/from JSON string using the ToJsonString()/FromJsonString() methods. In addition, a FieldMask message has the following methods:

• IsValidForDescriptor(message_descriptor): Checks whether the FieldMask is valid for Message Descriptor.
• AllFieldsFromDescriptor(message_descriptor): Gets all direct fields of Message Descriptor to FieldMask.
• CanonicalFormFromMask(mask): Converts a FieldMask to the canonical form.
• Union(mask1, mask2): Merges two FieldMasks into this FieldMask.
• Intersect(mask1, mask2): Intersects two FieldMasks into this FieldMask.
• MergeMessage(source, destination, replace_message_field=False, replace_repeated_field=False): Merges fields specified in FieldMask from source to destination.

Struct

Struct messages let you get and set the items directly. For example:

struct_message["key1"] = 5
struct_message["key2"] = "abc"
struct_message["key3"] = True

To get or create a list/struct, you can call get_or_create_list()/get_or_create_struct(). For example:

struct.get_or_create_struct("key4")["subkey"] = 11.0
struct.get_or_create_list("key5")

ListValue

A ListValue message acts like a Python sequence that lets you do the following:

list_value = struct_message.get_or_create_list("key")
list_value.extend([6, "seven", True, None])
list_value.append(False)
assert len(list_value) == 5
assert list_value[0] == 6
assert list_value[1] == "seven"
assert list_value[2] == True
assert list_value[3] == None
assert list_Value[4] == False

To add a ListValue/Struct, call add_list()/add_struct(). For example:

list_value.add_struct()["key"] = 1
list_value.add_list