Skip to content

python_message

Contains a metaclass and helper functions used to create protocol message classes from Descriptor objects at runtime.

Recall that a metaclass is the "type" of a class. (A class is to a metaclass what an instance is to a class.)

In this case, we use the GeneratedProtocolMessageType metaclass to inject all the useful functionality into the classes output by the protocol compiler at compile-time.

The upshot of all this is that the real implementation details for ALL pure-Python protocol buffers are here in this file.

GeneratedProtocolMessageType

Bases: type

Metaclass for protocol message classes created at runtime from Descriptors.

We add implementations for all methods described in the Message class. We also create properties to allow getting/setting all fields in the protocol message. Finally, we create slots to prevent users from accidentally "setting" nonexistent fields in the protocol message, which then wouldn't get serialized / deserialized properly.

The protocol compiler currently uses this metaclass to create protocol message classes at runtime. Clients can also manually create their own classes at runtime, as in this example:

mydescriptor = Descriptor(.....) factory = symbol_database.Default() factory.pool.AddDescriptor(mydescriptor) MyProtoClass = factory.GetPrototype(mydescriptor) myproto_instance = MyProtoClass() myproto.foo_field = 23 ...

Source code in client/ayon_hiero/vendor/google/protobuf/internal/python_message.py
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
class GeneratedProtocolMessageType(type):

  """Metaclass for protocol message classes created at runtime from Descriptors.

  We add implementations for all methods described in the Message class.  We
  also create properties to allow getting/setting all fields in the protocol
  message.  Finally, we create slots to prevent users from accidentally
  "setting" nonexistent fields in the protocol message, which then wouldn't get
  serialized / deserialized properly.

  The protocol compiler currently uses this metaclass to create protocol
  message classes at runtime.  Clients can also manually create their own
  classes at runtime, as in this example:

  mydescriptor = Descriptor(.....)
  factory = symbol_database.Default()
  factory.pool.AddDescriptor(mydescriptor)
  MyProtoClass = factory.GetPrototype(mydescriptor)
  myproto_instance = MyProtoClass()
  myproto.foo_field = 23
  ...
  """

  # Must be consistent with the protocol-compiler code in
  # proto2/compiler/internal/generator.*.
  _DESCRIPTOR_KEY = 'DESCRIPTOR'

  def __new__(cls, name, bases, dictionary):
    """Custom allocation for runtime-generated class types.

    We override __new__ because this is apparently the only place
    where we can meaningfully set __slots__ on the class we're creating(?).
    (The interplay between metaclasses and slots is not very well-documented).

    Args:
      name: Name of the class (ignored, but required by the
        metaclass protocol).
      bases: Base classes of the class we're constructing.
        (Should be message.Message).  We ignore this field, but
        it's required by the metaclass protocol
      dictionary: The class dictionary of the class we're
        constructing.  dictionary[_DESCRIPTOR_KEY] must contain
        a Descriptor object describing this protocol message
        type.

    Returns:
      Newly-allocated class.

    Raises:
      RuntimeError: Generated code only work with python cpp extension.
    """
    descriptor = dictionary[GeneratedProtocolMessageType._DESCRIPTOR_KEY]

    if isinstance(descriptor, str):
      raise RuntimeError('The generated code only work with python cpp '
                         'extension, but it is using pure python runtime.')

    # If a concrete class already exists for this descriptor, don't try to
    # create another.  Doing so will break any messages that already exist with
    # the existing class.
    #
    # The C++ implementation appears to have its own internal `PyMessageFactory`
    # to achieve similar results.
    #
    # This most commonly happens in `text_format.py` when using descriptors from
    # a custom pool; it calls symbol_database.Global().getPrototype() on a
    # descriptor which already has an existing concrete class.
    new_class = getattr(descriptor, '_concrete_class', None)
    if new_class:
      return new_class

    if descriptor.full_name in well_known_types.WKTBASES:
      bases += (well_known_types.WKTBASES[descriptor.full_name],)
    _AddClassAttributesForNestedExtensions(descriptor, dictionary)
    _AddSlots(descriptor, dictionary)

    superclass = super(GeneratedProtocolMessageType, cls)
    new_class = superclass.__new__(cls, name, bases, dictionary)
    return new_class

  def __init__(cls, name, bases, dictionary):
    """Here we perform the majority of our work on the class.
    We add enum getters, an __init__ method, implementations
    of all Message methods, and properties for all fields
    in the protocol type.

    Args:
      name: Name of the class (ignored, but required by the
        metaclass protocol).
      bases: Base classes of the class we're constructing.
        (Should be message.Message).  We ignore this field, but
        it's required by the metaclass protocol
      dictionary: The class dictionary of the class we're
        constructing.  dictionary[_DESCRIPTOR_KEY] must contain
        a Descriptor object describing this protocol message
        type.
    """
    descriptor = dictionary[GeneratedProtocolMessageType._DESCRIPTOR_KEY]

    # If this is an _existing_ class looked up via `_concrete_class` in the
    # __new__ method above, then we don't need to re-initialize anything.
    existing_class = getattr(descriptor, '_concrete_class', None)
    if existing_class:
      assert existing_class is cls, (
          'Duplicate `GeneratedProtocolMessageType` created for descriptor %r'
          % (descriptor.full_name))
      return

    cls._decoders_by_tag = {}
    if (descriptor.has_options and
        descriptor.GetOptions().message_set_wire_format):
      cls._decoders_by_tag[decoder.MESSAGE_SET_ITEM_TAG] = (
          decoder.MessageSetItemDecoder(descriptor), None)

    # Attach stuff to each FieldDescriptor for quick lookup later on.
    for field in descriptor.fields:
      _AttachFieldHelpers(cls, field)

    descriptor._concrete_class = cls  # pylint: disable=protected-access
    _AddEnumValues(descriptor, cls)
    _AddInitMethod(descriptor, cls)
    _AddPropertiesForFields(descriptor, cls)
    _AddPropertiesForExtensions(descriptor, cls)
    _AddStaticMethods(cls)
    _AddMessageMethods(descriptor, cls)
    _AddPrivateHelperMethods(descriptor, cls)

    superclass = super(GeneratedProtocolMessageType, cls)
    superclass.__init__(name, bases, dictionary)

__init__(name, bases, dictionary)

Here we perform the majority of our work on the class. We add enum getters, an init method, implementations of all Message methods, and properties for all fields in the protocol type.

Parameters:

Name Type Description Default
name

Name of the class (ignored, but required by the metaclass protocol).

required
bases

Base classes of the class we're constructing. (Should be message.Message). We ignore this field, but it's required by the metaclass protocol

required
dictionary

The class dictionary of the class we're constructing. dictionary[_DESCRIPTOR_KEY] must contain a Descriptor object describing this protocol message type.

required
Source code in client/ayon_hiero/vendor/google/protobuf/internal/python_message.py
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def __init__(cls, name, bases, dictionary):
  """Here we perform the majority of our work on the class.
  We add enum getters, an __init__ method, implementations
  of all Message methods, and properties for all fields
  in the protocol type.

  Args:
    name: Name of the class (ignored, but required by the
      metaclass protocol).
    bases: Base classes of the class we're constructing.
      (Should be message.Message).  We ignore this field, but
      it's required by the metaclass protocol
    dictionary: The class dictionary of the class we're
      constructing.  dictionary[_DESCRIPTOR_KEY] must contain
      a Descriptor object describing this protocol message
      type.
  """
  descriptor = dictionary[GeneratedProtocolMessageType._DESCRIPTOR_KEY]

  # If this is an _existing_ class looked up via `_concrete_class` in the
  # __new__ method above, then we don't need to re-initialize anything.
  existing_class = getattr(descriptor, '_concrete_class', None)
  if existing_class:
    assert existing_class is cls, (
        'Duplicate `GeneratedProtocolMessageType` created for descriptor %r'
        % (descriptor.full_name))
    return

  cls._decoders_by_tag = {}
  if (descriptor.has_options and
      descriptor.GetOptions().message_set_wire_format):
    cls._decoders_by_tag[decoder.MESSAGE_SET_ITEM_TAG] = (
        decoder.MessageSetItemDecoder(descriptor), None)

  # Attach stuff to each FieldDescriptor for quick lookup later on.
  for field in descriptor.fields:
    _AttachFieldHelpers(cls, field)

  descriptor._concrete_class = cls  # pylint: disable=protected-access
  _AddEnumValues(descriptor, cls)
  _AddInitMethod(descriptor, cls)
  _AddPropertiesForFields(descriptor, cls)
  _AddPropertiesForExtensions(descriptor, cls)
  _AddStaticMethods(cls)
  _AddMessageMethods(descriptor, cls)
  _AddPrivateHelperMethods(descriptor, cls)

  superclass = super(GeneratedProtocolMessageType, cls)
  superclass.__init__(name, bases, dictionary)

__new__(name, bases, dictionary)

Custom allocation for runtime-generated class types.

We override new because this is apparently the only place where we can meaningfully set slots on the class we're creating(?). (The interplay between metaclasses and slots is not very well-documented).

Parameters:

Name Type Description Default
name

Name of the class (ignored, but required by the metaclass protocol).

required
bases

Base classes of the class we're constructing. (Should be message.Message). We ignore this field, but it's required by the metaclass protocol

required
dictionary

The class dictionary of the class we're constructing. dictionary[_DESCRIPTOR_KEY] must contain a Descriptor object describing this protocol message type.

required

Returns:

Type Description

Newly-allocated class.

Raises:

Type Description
RuntimeError

Generated code only work with python cpp extension.

Source code in client/ayon_hiero/vendor/google/protobuf/internal/python_message.py
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
def __new__(cls, name, bases, dictionary):
  """Custom allocation for runtime-generated class types.

  We override __new__ because this is apparently the only place
  where we can meaningfully set __slots__ on the class we're creating(?).
  (The interplay between metaclasses and slots is not very well-documented).

  Args:
    name: Name of the class (ignored, but required by the
      metaclass protocol).
    bases: Base classes of the class we're constructing.
      (Should be message.Message).  We ignore this field, but
      it's required by the metaclass protocol
    dictionary: The class dictionary of the class we're
      constructing.  dictionary[_DESCRIPTOR_KEY] must contain
      a Descriptor object describing this protocol message
      type.

  Returns:
    Newly-allocated class.

  Raises:
    RuntimeError: Generated code only work with python cpp extension.
  """
  descriptor = dictionary[GeneratedProtocolMessageType._DESCRIPTOR_KEY]

  if isinstance(descriptor, str):
    raise RuntimeError('The generated code only work with python cpp '
                       'extension, but it is using pure python runtime.')

  # If a concrete class already exists for this descriptor, don't try to
  # create another.  Doing so will break any messages that already exist with
  # the existing class.
  #
  # The C++ implementation appears to have its own internal `PyMessageFactory`
  # to achieve similar results.
  #
  # This most commonly happens in `text_format.py` when using descriptors from
  # a custom pool; it calls symbol_database.Global().getPrototype() on a
  # descriptor which already has an existing concrete class.
  new_class = getattr(descriptor, '_concrete_class', None)
  if new_class:
    return new_class

  if descriptor.full_name in well_known_types.WKTBASES:
    bases += (well_known_types.WKTBASES[descriptor.full_name],)
  _AddClassAttributesForNestedExtensions(descriptor, dictionary)
  _AddSlots(descriptor, dictionary)

  superclass = super(GeneratedProtocolMessageType, cls)
  new_class = superclass.__new__(cls, name, bases, dictionary)
  return new_class