Does python preserve order of annotations with other methods?

Question

Consider the following class:

@dataclass
class Point:
   id: int

   x: int
   y: int
   @property
   def distance_from_zero(self): return (self.x**2 + self.y**2)**0.5

   color: tuple #RGB or something...

I know that I can get the annotations from the __annotations__ variable, or the fields function from dataclasses.fields and in order. I also know that the normal methods of any object can be read with dir or using the __dict__ method.

But what I'm after is something that can give me both in the right order, in the above case, it'll be something like:

>>>get_all_fields(Point)
['id', 'x', 'y', 'distance_from_zero', 'color']

The only thing i can think of is using something like the inspect module to read the actual code and somehow find the order. But that sounds really nasty.

Answer 1

There is also a solution involving metaclasses and the __prepare__ hook. The trick is to use it to provide a custom dictionary that will be used throughout the class creation, keeping track of new additions in a fields entry, and which propagates this accumulation of fields also when __annotations__ is added to it.

class _FieldAccumulatorDict(dict):

    def __init__(self, fields=None):
        if fields is None:
            # class dict, add a new `fields` list to the dictionary
            fields = []
            super().__setitem__("fields", fields)
        self.__fields = fields

    def __setitem__(self, key, value):
        if not key.startswith("__"):
            self.__fields.append(key)
        elif key == "__annotations__":
            # propagate accumulation when the `__annotation__` field is set internally
            value = _FieldAccumulatorDict(self.__fields)
        super().__setitem__(key, value)


class FieldAccumulatorMetaclass(type):

    def __prepare__(metacls, *args, **kwargs):
        return _FieldAccumulatorDict()


@dataclass
class Point(metaclass=FieldAccumulatorMetaclass):
    id: int

    x: int
    y: int

    @property
    def distance_from_zero(self): return (self.x ** 2 + self.y ** 2) ** 0.5

    color: tuple  # RGB or something...


print(Point.fields)  # will print ['id', 'x', 'y', 'distance_from_zero', 'color']

Consider also that metaclasses are inherited, so the metaclass can be assigned just to the root class and will then be used by all classes in the hierarchy.

Answer 2

It is possible to build a clean solution with introspection, since python is nice enough to make the module in charge of parsing python code (ast) available to us.

The api takes some getting used to, but here is a fix for your problem with inspect.getsource and a little custom ast-node walker:

import ast
import inspect


class AttributeVisitor(ast.NodeVisitor):
    def visit_ClassDef(self, node):
        self.attributes = []
        for statement in node.body:
            if isinstance(statement, ast.AnnAssign):
                self.attributes.append(statement.target.id)
            elif isinstance(statement, ast.FunctionDef):
                # only consider properties
                if statement.decorator_list:
                    if "property" in [d.id for d in statement.decorator_list]:
                        self.attributes.append(statement.name)
            else:
                print(f"Skipping {statement=}")

# parse the source code of "Point", so we don't have to write a parser ourselves
tree = ast.parse(inspect.getsource(Point), '<string>')

# create a visitor and run it over the tree line by line
visitor = AttributeVisitor()
visitor.visit(tree)

# print result, should be ['id', 'x', 'y', 'distance_from_zero', 'color']
print(visitor.attributes)

Using this solution means you don't have to alter your Point class in any way to get what you want.