pyguides

Understanding Descriptors in Python

What Python descriptors actually do

If you’ve ever wondered how obj.attr can run validation, log access, compute a value lazily, or behave differently at the class level than at the instance level, the answer is almost always the same: descriptors. The descriptor protocol is the machinery Python uses to customize attribute access, and it quietly powers @property, @classmethod, @staticmethod, bound methods, ORM model fields, and functools.cached_property. Once you see it, a lot of Python stops feeling magical and starts feeling mechanical.

A descriptor is any object that defines at least one of the four protocol methods: __get__, __set__, __delete__, or __set_name__. When such an object is stored as a class attribute, Python’s normal attribute lookup detects it and hands the access over to the descriptor’s methods. That single mechanism is what makes “managed attributes” possible.

The descriptor protocol

The four protocol methods, exactly as the Python data model defines them:

object.__get__(self, instance, owner=None)
object.__set__(self, instance, value)
object.__delete__(self, instance)
object.__set_name__(self, owner, name)   # added in Python 3.6

A few details worth keeping in your head:

  • __get__ is called for both instance and class access. When you read A.x on the class itself, instance is None and owner is A. When you read a.x on an instance, instance is a and owner is type(a).
  • __get__ must return the computed value or raise AttributeError. Raising AttributeError is the documented way to say “I don’t have a value to give you.”
  • __set__ and __delete__ mutate or remove the attribute on instance. They should not return anything.
  • __set_name__ runs automatically when the descriptor is assigned inside a class body. It receives the owning class and the attribute name, which means the descriptor doesn’t have to be told what it’s called. If you assign the descriptor after the class is created, you have to call __set_name__ yourself.

The smallest possible descriptor looks like this, taken almost verbatim from the standard library’s descriptor guide:

class Ten:
    def __get__(self, obj, objtype=None):
        return 10

class A:
    x = 5
    y = Ten()

a = A()
a.x   # 5
a.y   # 10

a.y is 10 even though nothing is stored anywhere. The value is recomputed on every access, which is the whole point of a descriptor that overrides __get__.

Data descriptors vs non-data descriptors

This is the single most important distinction in the protocol.

A data descriptor defines __set__ and/or __delete__. A non-data descriptor defines only __get__. The difference matters because data descriptors always win the lookup race against the instance’s __dict__, while non-data descriptors can be shadowed by instance attributes.

Concretely:

  • property is a data descriptor. That’s why you can’t accidentally add an instance attribute with the same name as a property, since the property’s __set__ (which raises AttributeError for read-only properties) intercepts the assignment before __dict__ ever sees it.
  • A plain function is a non-data descriptor. That’s how bound methods work, and it’s also why you can do instance.method = some_callable to monkey-patch a single instance without affecting the rest.
  • classmethod and staticmethod are non-data descriptors. They define __get__ to return either a bound cls method or the unwrapped function.

A handy trick: to make a read-only attribute, define both __get__ and __set__, and have __set__ raise AttributeError. The presence of __set__ is what makes it a data descriptor; the raising body is what makes it read-only. That’s exactly what property(fget) does under the hood.

How attribute lookup actually walks the chain

For an expression like obj.x, Python checks, in this order:

  1. Data descriptors on type(obj) and its MRO, in the class and its base classes.
  2. The instance’s own __dict__.
  3. Non-data descriptors on type(obj) and its MRO.
  4. __getattr__ on the class, if defined.

The practical consequence: a data descriptor’s value can’t be overridden by setting obj.x = ..., but a non-data descriptor can be. You can verify this with a few lines:

class Data:
    def __get__(self, obj, objtype=None): return 'data-get'
    def __set__(self, obj, value): print('data-set called')

class NonData:
    def __get__(self, obj, objtype=None): return 'non-data-get'

class C:
    d = Data()
    n = NonData()

c = C()
c.d = 99          # data-set called     (descriptor wins)
c.__dict__['d']   # KeyError
c.n = 99          # no output           (instance dict wins)
c.__dict__['n']   # 99

The instance dict silently shadowed the non-data descriptor, while the data descriptor caught the assignment and never touched the instance dict at all.

A reusable validator descriptor

The classic real-world use of descriptors is validation. Here’s a framework that builds on what the standard library’s descriptor howto shows: an abstract Validator base class and a few concrete validators that you can mix and match on any class.

from abc import ABC, abstractmethod

class Validator(ABC):
    def __set_name__(self, owner, name):
        self.private_name = '_' + name

    def __get__(self, obj, objtype=None):
        return getattr(obj, self.private_name)

    def __set__(self, obj, value):
        self.validate(value)
        setattr(obj, self.private_name, value)

    @abstractmethod
    def validate(self, value):
        pass

class OneOf(Validator):
    def __init__(self, *options):
        self.options = set(options)
    def validate(self, value):
        if value not in self.options:
            raise ValueError(f'Expected {value!r} to be one of {sorted(self.options)}')

class Number(Validator):
    def __init__(self, minvalue=None, maxvalue=None):
        self.minvalue, self.maxvalue = minvalue, maxvalue
    def validate(self, value):
        if not isinstance(value, (int, float)):
            raise TypeError(f'Expected {value!r} to be an int or float')
        if self.minvalue is not None and value < self.minvalue:
            raise ValueError(f'Expected {value!r} to be at least {self.minvalue!r}')
        if self.maxvalue is not None and value > self.maxvalue:
            raise ValueError(f'Expected {value!r} to be at most {self.maxvalue!r}')

class String(Validator):
    def __init__(self, minsize=None, maxsize=None, predicate=None):
        self.minsize, self.maxsize, self.predicate = minsize, maxsize, predicate
    def validate(self, value):
        if not isinstance(value, str):
            raise TypeError(f'Expected {value!r} to be a str')
        if self.minsize is not None and len(value) < self.minsize:
            raise ValueError(f'Expected {value!r} to be at least {self.minsize} chars')
        if self.maxsize is not None and len(value) > self.maxsize:
            raise ValueError(f'Expected {value!r} to be at most {self.maxsize} chars')
        if self.predicate is not None and not self.predicate(value):
            raise ValueError(f'Expected {self.predicate.__name__} to be true for {value!r}')

Using it on a class is short and reads naturally. The Component class below wires up three validators: a name that must be uppercase and between 3 and 10 characters, a kind chosen from a fixed set of strings, and a non-negative numeric quantity. The lines that follow it show what each broken input produces and which validator fires.

class Component:
    name = String(minsize=3, maxsize=10, predicate=str.isupper)
    kind = OneOf('wood', 'metal', 'plastic')
    quantity = Number(minvalue=0)

    def __init__(self, name, kind, quantity):
        self.name = name
        self.kind = kind
        self.quantity = quantity

c = Component('Widget', 'metal', 5)        # ValueError: Expected isupper to be true for 'Widget'
c = Component('WIDGET', 'metle', 5)        # ValueError: Expected 'metle' to be one of ['metal', 'plastic', 'wood']
c = Component('WIDGET', 'metal', -5)       # ValueError: Expected -5 to be at least 0
c = Component('WIDGET', 'metal', 'V')      # TypeError: Expected 'V' to be an int or float
c = Component('WIDGET', 'metal', 5)        # works

Two things to notice. First, __set_name__ does the bookkeeping so the descriptor knows to store its value at _name, _kind, and _quantity (no per-class configuration). Second, because every Validator defines __set__, each one is a data descriptor, which is exactly what you want for validation. The instance dict never silently accepts an invalid value.

Lazy and cached attributes

Descriptors also make it easy to compute an attribute once and then cache the result. Here is a hand-rolled lazy property, followed by a quick look at the standard library equivalent.

class Lazy:
    def __init__(self, func):
        self.func = func
        self.name = func.__name__

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        value = self.func(obj)
        setattr(obj, self.name, value)   # replaces the descriptor on this instance
        return value

class Circle:
    def __init__(self, radius):
        self.radius = radius

    @Lazy
    def area(self):
        import math
        print('computing area')
        return math.pi * self.radius ** 2

c = Circle(2)
c.area   # computing area / 12.566370614359172
c.area   # 12.566370614359172  (no recompute)

The setattr on the second access is what makes this work: by writing a real number into the instance dict under area, the descriptor is shadowed on that one instance. Since Lazy is a non-data descriptor (it has no __set__), the instance dict wins and subsequent reads skip the descriptor entirely. For most projects, you should just use functools.cached_property instead; it does the same thing, handles thread safety, and is a data descriptor so it works correctly with inheritance.

How the built-ins you already use are descriptors

The payoff for understanding all of this is that the language features you use every day fall out of the same protocol.

  • property: a data descriptor. It defines __get__, __set__, and __delete__, which is why instance attributes can’t shadow it.
  • classmethod: a non-data descriptor whose __get__ returns a method bound to the class, not the instance.
  • staticmethod: a non-data descriptor whose __get__ returns the wrapped function unchanged, skipping the binding step entirely.
  • Plain functions: non-data descriptors. obj.method resolves to function.__get__(obj, type(obj)), which is exactly the magic that produces a bound method.
  • functools.cached_property: a data descriptor, since 3.8. Behaves like a property whose value is computed once.
  • __slots__: implemented at the class level by creating a descriptor for each slot name, which is why slotted classes don’t get an instance __dict__ and use less memory.

If you remember nothing else, remember this: a method on an instance is just a function whose __get__ happened to bind it. There is no separate “method” type in Python’s data model. The binding is a property of the descriptor protocol.

Common pitfalls

A few things that catch people the first time they write descriptors:

  1. Descriptors only work as class variables. Putting a descriptor in an instance’s __dict__ has no effect, because the protocol only triggers when attribute lookup walks the class.
  2. The descriptor object is shared across all instances. Don’t store per-instance data on self inside the descriptor. Store class-level config (limits, allowed values) on self, and per-instance data in the instance’s __dict__ under a private name.
  3. __set_name__ only fires for class-body assignments. If you build the class dynamically or assign the descriptor later, call __set_name__ by hand: descriptor.__set_name__(MyClass, 'field_name').
  4. hasattr follows the same protocol. If your descriptor’s __get__ always returns a truthy value, hasattr(obj, 'name') will be True even when no value has been set. Raise AttributeError to signal “no value.”
  5. A property with only a getter is still a data descriptor. Even though it raises on assignment, the presence of __set__ is what prevents instance-shadowing. This is by design.
  6. __getattr__ is not a descriptor. It runs only when normal attribute lookup fails. If you find yourself writing __getattr__ to compute a value, you probably wanted a descriptor instead.
  7. Don’t return None from __get__ on a class-level access to mean “no value.” Either raise AttributeError or return a real sentinel. The protocol explicitly supports AttributeError; everything else is your caller’s problem.

Once those land, the descriptor pattern stops feeling exotic. It becomes the natural way to put a tiny piece of code between “the user wrote obj.x” and “the value the user actually gets back.”

See Also