Overview

Currently, the following meta-classes are provided:

• ExtendedType - slotted types, mixin-classes, singletons, abstract classes and expected members, in any combination.

See also

Meta Classes

Understanding Python metaclasses

Python Data Model

General Data Model of Python and section about __slots__.

ExtendedType

The new meta-class ExtendedType allows to implement singletons, slotted types and combinations thereof.

Since Python 3, meta-classes are applied in a class definition by adding a named parameter called metaclass to the list of derived classes (positional parameters). Further named parameters might be given to pass parameters to that new meta-class.

class MyClass(metaclass=ExtendedType):
  pass

Each feature is switched on by a class keyword argument:

Keyword	Effect	Section
slots=True	Store the annotated fields in __slots__ instead of a __dict__.	Slotted
mixin=True	Collect the annotated fields, and add them to the slots of the class the mixin-class is mixed into.	Mixin
weakref=True	Let instances of a slotted class be referenced weakly, by adding __weakref__ to its slots.	Weak References
singleton=True	Create one instance, and return it on every further instantiation.	Singleton
expects=(...)	Name the members the class needs from the class it is mixed into.	Expected Members

Abstract classes and methods need no keyword: a decorator marks them, and ExtendedType rejects instantiating the class - see Abstract Class, Abstract Method and MustOverwrite Method.

See also

Meta-Classes

→ What a meta-class is, what to try before writing one, and why slots need one.

Slotted

slots=True stores an instance’s fields in __slots__ instead of a __dict__. Every annotated field of the class body becomes a slot. A class variable is annotated as ClassVar and stays a class variable.

Example:

from __future__            import annotations
from typing                import ClassVar
from pyTooling.MetaClasses import ExtendedType

class Node(metaclass=ExtendedType, slots=True):
  _SEPARATOR: ClassVar[str] = "/"   # class variable
  _parent:    Node                  # slot
  _name:      str                   # slot

  def __init__(self, parent: Node, name: str) -> None:
    self._parent = parent
    self._name =   name

Node.__slots__                      # ('_parent', '_name')
Node(None, "root")._nmae = "x"      # AttributeError: no __dict__ for setting new attributes

An instance has no __dict__, so it needs less memory and reads its fields faster, and an assignment to a misspelt field raises an AttributeError instead of creating a new field.

Initial values

A slot can’t have a class attribute of the same name, so an initial value in the class body (_name: str = "root") can’t stay there. ExtendedType moves it into an intermediate __init__, which assigns the initial values of the whole hierarchy to the new instance and then calls the class’ own __init__:

class Node(metaclass=ExtendedType, slots=True):
  _name:  str = "root"
  _count: int = 0

  def __init__(self, count: int) -> None:
    self._count = count

Node(3)._name                       # 'root'
Node(3)._count                      # 3

• The initial values are assigned before any __init__ runs, so a derived class’ __init__ may assign a base-class’ field before calling super().__init__().

• A base-class’ initial values are assigned too, also if a derived class’ __init__ doesn’t call super().__init__().

• The class a mixin-class is mixed into assigns the mixin-class’ initial values too.

• The value is assigned, not copied: a mutable initial value ([]) is one object shared by all instances, like a class variable. Assign a new one in __init__ instead.

• A class variable (ClassVar) keeps its initial value on the class.

Rules

• Every base-class uses slots. A base-class without __slots__ would give each instance a __dict__ anyway: BaseClassWithoutSlotsError.

• Derived classes are slotted too. A class derived from a slotted class lists only its own new fields in __slots__; __allSlots__ holds the fields of the whole hierarchy.

• A field is declared once in a hierarchy. Annotating a base-class’ field again raises an AttributeError. Assigning it in the class body without an annotation would hide the slot, and raises a DuplicateFieldInSlotsError.

• Only one base-class has non-empty slots. That is Python’s rule; every further base-class is a mixin-class.

• Every field is annotated. A field assigned in the class body without an annotation is neither a slot nor a declared class variable. It stays a class attribute, and ExtendedType reports an UnannotatedFieldWarning to the innermost WarningCollector; without a collector, it’s dropped.

Serialization

A slotted class gets __getstate__ and __setstate__, unless it defines them itself, so its instances work with pickle and copy. The state is a dictionary of every slot in the hierarchy.

• Pickling an instance with an unassigned slot raises an ExtendedTypeError naming the field.

• Unpickling a state with a missing or an unexpected field raises an ExtendedTypeError naming them.

SlottedObject

Deriving from SlottedObject makes a class slotted without naming the meta-class, and the @slotted decorator recreates a class as a slotted one. All three spellings declare the same class:

Deriving from SlottedObject	Apply slotted Decorator	Using meta-class ExtendedType
class MyClass(SlottedObject): pass	@slotted class MyClass: pass	class MyClass(metaclass=ExtendedType, slots=True): pass

Mixin

Python allows non-empty __slots__ on one inheritance line only. mixin=True declares a mixin-class: its annotated fields are collected in __mixinSlots__ instead of becoming slots, and they become slots of the slotted class the mixin-class is mixed into.

Example:

from __future__            import annotations
from pyTooling.MetaClasses import ExtendedType

class NamedMixin(metaclass=ExtendedType, mixin=True):
  _name: str

  def __init__(self, name: str) -> None:
    self._name = name

class BaseNode(metaclass=ExtendedType, slots=True):
  _parent: Node

  def __init__(self, parent: Node) -> None:
    self._parent = parent

class Node(BaseNode, NamedMixin):
  def __init__(self, parent: Node, name: str) -> None:
    super().__init__(parent)
    NamedMixin.__init__(self, name)

NamedMixin.__slots__                # ()
NamedMixin.__mixinSlots__           # ('_name',)
Node.__slots__                      # ('_name',)
Node.__allSlots__                   # {'_parent', '_name'}

The mixin-class’ __init__ sets its own fields; the class it is mixed into calls it after its primary base-class’.

Rules

• The primary base-class comes first. The first base-class is the primary inheritance line and carries the slots; every further base-class is a mixin-class.

• A further base-class with slots is rejected. A slotted class that isn’t a mixin-class raises a BaseClassIsNotAMixinError, also with empty slots. Another class with non-empty __slots__ raises a BaseClassWithNonEmptySlotsError; a class without __slots__ at all raises a BaseClassWithoutSlotsError.

• Mixin-classes combine. A mixin-class derived from other mixin-classes collects their fields too, and a mixin-class may derive from a class of the primary inheritance line, e.g. to use its members.

• A mixin-class is used mixed in. An instance of the mixin-class itself has neither slots for its fields nor a __dict__, so it can’t hold them.

• Without slots, fields stay attributes. Mixed into a class that doesn’t use slots, the mixin-class’ fields are ordinary attributes in the instance’s __dict__.

• The host class doesn’t shadow a mixin field. Assigning a member of that name in the class body raises a DuplicateFieldInSlotsError.

• A mixin field exists once. The host class annotating it again, a base-class of the primary inheritance line having a slot of that name, or a second mixin-class declaring it raises a DuplicateFieldInSlotsError. Two mixin-classes deriving from one mixin-class pass its field on twice; it becomes one slot.

The @mixin decorator recreates a class as a mixin-class. A mixin-class names the members it needs from the class it is mixed into with expects.

Weak References

A class using __slots__ cannot be referenced weakly. There is no __dict__ for weakref to put its bookkeeping in, and the slot that would hold it is not created unless it is asked for:

class Slotted(metaclass=ExtendedType, slots=True):
  _field: int

weakref.ref(Slotted())     # TypeError: cannot create weak reference to 'Slotted' object

weakref=True adds __weakref__ to the class’ slots, which is what makes an instance weak-referenceable:

Example:

from __future__            import annotations
from weakref               import ref
from pyTooling.MetaClasses import ExtendedType

class Node(metaclass=ExtendedType, slots=True, weakref=True):
  _parent: Node

node = Node()
reference = ref(node)      # fine

reference()                # -> the node
del node
reference()                # -> None

The typical reason to want it is a back-reference: a child pointing at its parent keeps the parent alive as long as the child lives, and a weak reference is how that cycle is avoided.

Important

__weakref__ is a slot like any other, and like any other it may appear once in an inheritance hierarchy. A derived class inherits the capability; asking for it again declares a duplicate slot:

class Base(metaclass=ExtendedType, slots=True, weakref=True): ...
class Derived(Base): ...                       # already weak-referenceable
class Again(Base, weakref=True): ...           # AttributeError: slot '__weakref__' already exists

A mixin-class can ask for it too. __weakref__ is then one of the slots it contributes, and it is added to the class the mixin-class is mixed into. That class must not have it already - the second one raises a DuplicateFieldInSlotsError:

from __future__ import annotations

class ParentMixin(metaclass=ExtendedType, mixin=True, weakref=True):
  _parent: Node

class Node(metaclass=ExtendedType, slots=True):
  _name: str

class Child(Node, ParentMixin): ...               # weak-referenceable

Attention

weakref=True is the only way to ask for it. Annotating __weakref__ as a field raises an ExtendedTypeError, as does annotating __dict__: a __dict__ slot accepts any attribute on an instance again and gives up what slots save. A class that needs a __dict__ uses slots=False.

Note

A class without slots is weak-referenceable already, because Python adds __weakref__ itself. weakref=True is therefore only meaningful together with slots=True or mixin=True.

Note

The restriction is CPython’s. On PyPy every object is weak-referenceable whether or not the class uses slots, so weakref=True adds the slot but changes nothing observable there. Code written for both keeps the keyword: it is what makes the class work on CPython.

Expected Members

A class that is only complete once it is combined with another uses members it doesn’t define itself. Nothing states that contract, so the class that forgets one fails with an AttributeError on first access - somewhere else entirely, and only if that code path ever runs.

expects names those members, and it works in both directions:

Declared on	Meaning	Rejected
a class (expects=(...))	“whatever I am mixed into must provide these” - the mixin-class case	instantiating the combined class
a method (@expects)	“my class must provide these” - including a method on the primary inheritance line waiting for a mixin-class to contribute them	calling that method

The second is the one a mixin-class does not cover: a class on the primary inheritance line cannot declare the members class-wide, because it has to stay usable without the mixin. See A method expecting what a mixin-class contributes.

The class keyword argument names the members a class needs from whichever class it is mixed into.

Example:

from pyTooling.MetaClasses import ExtendedType

class ReportMixin(metaclass=ExtendedType, mixin=True, expects=("_counter", "Write")):
  def Report(self) -> bool:
    return self.Write(f"{self._counter}")

class Application(TerminalApplication, ReportMixin):
  pass

Application()   # fine, if 'TerminalApplication' provides '_counter' and 'Write'

A member is provided when it is reachable on the class: a method, a property, a class variable, or a field, for which ExtendedType created a slot descriptor when the mixin joined the primary inheritance line. So a field declared as an annotation counts, and both kinds of member are covered by one list.

When something is missing, instantiating the class raises an UnfulfilledExpectationError naming every missing member and the class expecting it:

pyTooling.MetaClasses.UnfulfilledExpectationError: Class 'Application' doesn't provide every expected member.
Missing 'Write', expected by 'ReportMixin'.
A mixin-class names what it needs from its host class with the 'expects' class keyword argument.

Which members are missing is computed once, when the class is constructed, and kept in __missingMembers__; the exception is raised on instantiation. That is the same mechanism an abstract class uses, and it has the same consequence: a class may stay incomplete as long as nothing instantiates it, so an intermediate class can pass an expectation on to its own subclasses. A class that fulfills it again is instantiable, without the intermediate class having to say anything.

expects is not limited to mixin-classes - any class can state what its subclasses have to provide. A mixin-class is the case it exists for, and a mixin-class is never itself incomplete, because it cannot provide what it expects from its host.

A method expecting what a mixin-class contributes

Sometimes only one method needs what a mixin-class contributes, while the class itself is perfectly usable without it. The marked method sits on the class in the primary inheritance line, and it waits for a mixin-class further along the bases to supply what it reads.

The class keyword argument would be too strict here: it would reject a class that never calls the method.

The decorator works from a mixin-class too, and pyTooling uses it that way: _PrintHelp() has the parsers - they are the mixin’s own fields - but it writes through the Write*** methods of TerminalApplication, so it expects those. Everything else the mixin does works without a terminal. Which side declares the expectation is decided by which side owns the method, not by which side is the mixin.

The @expects decorator moves the expectation to where it belongs.

Example:

from pyTooling.MetaClasses import ExtendedType, expects

class Terminal(metaclass=ExtendedType, slots=True):
  @expects("MainParser", "SubParsers")
  def PrintHelp(self) -> None:
    self.MainParser.print_help()

Terminal().PrintHelp()                  # UnfulfilledExpectationError

class Application(Terminal, ArgParseHelperMixin):
  pass

Application().PrintHelp()               # fine

The class stays usable - constructing it, instantiating it and calling every other method is unaffected. Only the method that cannot work is replaced, by one raising an UnfulfilledExpectationError that names the missing members:

UnfulfilledExpectationError: Method 'Terminal.PrintHelp()' expects members this class doesn't provide.
Missing 'MainParser'.
Missing 'SubParsers'.
A method names what it needs from its class with the 'expects' decorator.

The check runs per class, so a fulfilled expectation costs nothing: the replacement is not installed at all and the class holds the original function, with no per-call test. A replacement inherited from a base-class is removed again as soon as a class provides the members, and a subclass that provides only some of them reports exactly the ones still missing.

Because it is evaluated per class rather than once at the declaring class, the mixin-class may arrive any number of levels further down - and a sibling that does not mix it in still reports the missing members:

class Middle(Terminal):                       # still incomplete, still fine
  pass

class Application(Middle, ArgParseHelperMixin):
  pass

Application().PrintHelp()                     # fine
Terminal().PrintHelp()                        # UnfulfilledExpectationError

A member counts whether it is a method, a property, a class variable, or a field the mixin-class declared as an annotation - ExtendedType materialises those as slots when the mixin joins the primary inheritance line, which is what makes them visible to the check.

The expectation belongs to the method, so an override doesn’t inherit it: a subclass overriding the method without @expects expects nothing, and one marking its override expects what its own marker names.

See also

@abstractmethod

→ Mark a method as abstract, when the class itself declares what has to be overridden.

Abstract Class

A class containing an abstract method cannot be instantiated, but some classes have nothing to mark abstract and still exist only to be derived from - a base-class collecting shared infrastructure, for instance.

The @abstractclass decorator declares a class as abstract (inheritance tree inner node) without the need for abstract methods.

Example:

from pyTooling.MetaClasses import ExtendedType, abstractclass

@abstractclass
class Base(metaclass=ExtendedType):
  def Method(self) -> None:
    ...

class Derived(Base):
  pass

Derived()   # fine

The decorator sets __abstractClass__ on the class and recomputes __isAbstract__, which is the same computation ExtendedType runs for abstract methods.

The marker belongs to the decorated class alone: ExtendedType clears it on every class it creates, so a derived class is concrete again unless it is decorated itself or inherits an abstract method. The declaration therefore describes one class rather than a branch of the hierarchy.

See also

@abstractmethod

→ Mark a method as abstract, which makes its class abstract as a consequence.

Abstract Method

The @abstractmethod decorator marks a method as abstract. The original method gets replaced by a method raising a NotImplementedError. When a class containing abstract methods is instantiated, an AbstractClassError is raised.

Example:

class A(metaclass=ExtendedType):
  @abstractmethod
  def method(self) -> int:
    """Methods documentation."""

class B(A):
  @InheritDocString(A)
  def method(self) -> int:
    return 2

A class method, static method or property is marked the same way. The decorator is applied to the function, so it is listed below classmethod, staticmethod, property or @readonly:

class A(metaclass=ExtendedType):
  @classmethod
  @abstractmethod
  def Parse(cls, value: str) -> "A":
    """Create an instance from a string."""

  @readonly
  @abstractmethod
  def Name(self) -> str:
    """Read-only property to return the name."""

class B(A):
  @classmethod
  def Parse(cls, value: str) -> "B":
    return cls()

  @readonly
  def Name(self) -> str:
    return "B"

Hint

If the abstract method should contain code that should be called from an overriding method in a derived class, use the @mustoverride decorator.

MustOverwrite Method

The @mustoverride decorator marks a method as must override. When a class containing must override methods is instantiated, an MustOverrideClassError is raised.

In contrast to @abstractmethod, the method can still be called from a derived class implementing an overridden method.

Example:

class A(metaclass=ExtendedType):
  @mustoverride
  def method(self) -> int:
    """Methods documentation."""
    return 2

class B(A):
  @InheritDocString(A)
  def method(self) -> int:
    result = super().method()
    return result + 1

Hint

If the method contain no code and throw an exception when called, use the @abstractmethod decorator.

Singleton

A class defined with enabled singleton behavior implements the singleton design pattern, which allows only a single instance of that class to exist. If another instance is going to be created, a previously cached instance of that class will be returned.

class MyClass(metaclass=ExtendedType, singleton=True):
  pass

Example Usage

class Terminal(metaclass=ExtendedType, singleton=True):
  def __init__(self) -> None:
    pass

  def WriteLine(self, message):
    print(message)

Overloading

Attention

Method overloading is planned and not available yet: pyTooling.MetaClasses has no overloadable decorator, and a class with ExtendedType keeps the last of two methods of one name, as Python does. It needs a clear definition before overloading makes sense.

A decorator overloadable would mark methods of the same name as overloads of each other, and ExtendedType would collect them into one method. A call is dispatched to the overload whose parameters’ type annotations match the arguments. If none matches, OverloadResolutionError is raised.

Planned Usage

class A(metaclass=ExtendedType):
  value: int

  @overloadable
  def __init__(self, value: int = 0) -> None:
    self.value = value

  @overloadable
  def __init__(self, value: str) -> None:
    self.value = int(value)

a = A()
print(a.value)

b = A(3)
print(b.value)

c = A("42")
print(c.value)

Competing Solutions

ExtendedType generates no methods - no __init__, __repr__ or __eq__ - and validates no values. It shapes the class: slots computed from the annotations, including those of mixin-classes, singletons, abstract classes, and members a mixin-class expects from its host. The packages below generate methods or validate values, and share only parts of that.

dataclasses

Source: dataclasses of the standard library.

Disadvantages

• @dataclass(slots=True) creates the slots of one class. Two slotted dataclasses can’t be base-classes of the same class (multiple bases have instance lay-out conflict), so a slotted mixin-class isn’t possible.

• No singleton, no abstract class, no expected members.

Standoff

• The fields are taken from the annotations, as ExtendedType takes the slots, and weakref_slot=True matches weakref=True.

Advantages

• Generates __init__, __repr__, __eq__ and, on request, ordering, hashing and frozen instances.

• Part of the standard library.

attrs

Source: attrs, on PyPI as attrs.

Disadvantages

• @attrs.define creates slotted classes by default, with the same limit as a dataclass: two slotted base-classes conflict, so fields can’t be contributed by mixin-classes.

• No singleton, no abstract class, no expected members.

Advantages

• Generates the methods a dataclass generates, plus validators and converters per field.

• Adds __weakref__ to the slots by default.

pydantic

Source: pydantic, on PyPI as pydantic.

Disadvantages

• A model keeps its fields in a __dict__, so it isn’t a slotted type.

• Depends on pydantic-core, a compiled extension, and three more packages.

Standoff

• A different purpose: validating, converting and serializing data, e.g. from JSON.

Advantages

• Validates every field’s value against its annotation, and creates JSON schemas.

abc

Source: abc of the standard library.

Disadvantages

• A class without an abstract method is concrete; there is no counterpart to @abstractclass.

• The body of an abstract method stays callable. A method whose call has to fail, as one marked by @abstractmethod does, is written by hand.

• Instantiating an abstract class raises a TypeError, not an exception of its own.

Standoff

• abc.abstractmethod matches @mustoverride: the class can’t be instantiated until the method is overridden, and an overriding method can call it with super().

Advantages

• Part of the standard library, and isinstance() checks against classes registered as virtual subclasses.