pyguides

Python Decorators Deep Dive: From Closures to ParamSpec

This article picks up where the introductory /guides/python-decorators/ leaves off. If you have not written a wrapper function or used functools.wraps before, read that one first. Here, you will see what actually happens at the byte-code level when the interpreter hits @decorator, why the late-binding loop trap exists, and how to type-hint a parameterized decorator so mypy and your editor stay useful.

All code targets Python 3.10+. The ParamSpec section needs 3.10+; the functools.cache call needs 3.9+; functools.cached_property needs 3.8+.

Two-phase execution: what runs at import time vs. call time

A decorator is just a callable that takes a function and returns a function. The interpreter applies it in two distinct phases, and confusing the two is the source of half the decorator bugs in the wild.

Phase one runs at definition time, the moment the def statement finishes. The interpreter binds the function object, evaluates the decorator expression, calls it with the function, and binds the result to the original name. Phase two runs every time someone calls the decorated name.

def shout(func):
    print(f"decorating {func.__name__}")
    return func

@shout
def hello():
    return "hi"

# output:
# decorating hello

The print fires once, at import time, even though hello is never called. That is why decorators can do expensive work (caching, registering, building wrappers) up front without paying for it on each invocation. It is also why side effects in a decorator run during module import, which matters in tests, REPL sessions, and any framework that introspects the module before you expect.

Closures are the engine (and the late-binding trap)

A typical decorator closes over the wrapped function. Python captures the binding by reference, not by value, and that one fact is responsible for the most common decorator bug: the late-binding loop trap.

# BAD: every lambda returns the same value
funcs = [lambda: i for i in range(3)]
print([f() for f in funcs])

# output: [2, 2, 2]

The body of each lambda looks up i from the enclosing scope at call time, after the loop has finished and i has settled on its last value. The same trap shows up when you build a list of decorators in a loop:

def make_multiplier(n):
    return lambda x: x * n

# BAD: all multipliers are 4
multipliers = [make_multiplier(i) for i in range(3)]
print([m(10) for m in multipliers])

# output: [40, 40, 40]

The fix is to capture the value as a default argument, which binds at definition time, not at call time. Default arguments are evaluated exactly once when the function object is created, so each lambda in the list ends up holding its own private value of i rather than a shared reference to the loop variable:

funcs = [lambda i=i: i for i in range(3)]
print([f() for f in funcs])

# output: [0, 1, 2]

If you are building decorators in a loop, the same pattern applies. The closure captures the loop variable by reference, not by value. For more on how closures store state across calls, see /guides/python-closures/.

functools.wraps in depth

functools.wraps is a thin convenience over functools.update_wrapper. The two constants it uses are public and worth knowing:

  • WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__annotations__', '__doc__'), copied onto the wrapper by direct assignment.
  • WRAPPER_UPDATES = ('__dict__',), merged in place so the wrapper keeps its own extra attributes.

wraps does not copy __signature__. Instead, it sets __wrapped__ on the wrapper, and inspect.signature follows that chain by default (follow_wrapped=True since 3.5). That is why stacking decorated functions still produces the right signature, as long as every layer uses @wraps.

from functools import wraps
import inspect

def logged(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print(f"calling {func.__name__}")
        return func(*args, **kwargs)
    return wrapper

@logged
@logged
def add(a: int, b: int) -> int:
    return a + b

print(add.__name__)           # add
print(inspect.signature(add)) # (a: int, b: int) -> int
# output:
# add
# (a: int, b: int) -> int

Without wraps, add.__name__ would be wrapper and the signature would be (*args, **kwargs). The whole functools module is documented at /reference/modules/functools-module/.

Decorator factories: decorators that take arguments

A decorator factory is a function that returns a decorator. It is how you parameterize behavior: repeat(n=3) instead of repeat. The structure has three nested callables:

  1. Outer factory, which receives the parameter (n=3) and builds a closure over it.
  2. Middle decorator, which receives the function and builds a closure over it.
  3. Inner wrapper, which runs at call time and has access to all of the above.
from functools import wraps

def repeat(n=3):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            return [func(*args, **kwargs) for _ in range(n)]
        return wrapper
    return decorator

@repeat(n=4)
def roll():
    import random
    return random.randint(1, 6)

print(roll())

Notice the call site: @repeat(n=4) calls repeat once (returning decorator), which is then applied to roll. Compare that to @logged where there is no parameter. The decorator itself is the only callable in the chain, so the call site just writes the decorator name directly above the function.

A clean way to read a factory is “the user is configuring a wrapper.” Two layers of closure look like overkill for one parameter, but the same shape scales to logging levels, retry counts, cache backends, feature flags, and so on. The factories pattern shows up everywhere in real codebases.

Class-based decorators: stateful wrappers

A class with __call__ is a callable, so it is a decorator. The class form shines when the decorator needs to keep state, such as a call counter, a cache, or a list of registered handlers.

from functools import wraps

class CountCalls:
    def __init__(self, func):
        wraps(func)(self)            # copy metadata onto the instance
        self.func = func
        self.calls = 0

    def __call__(self, *args, **kwargs):
        self.calls += 1
        print(f"{self.func.__name__} called {self.calls} time(s)")
        return self.func(*args, **kwargs)

@CountCalls
def greet(name):
    return f"hello {name}"

greet("alice")
greet("bob")
print(greet.calls)

# output:
# greet called 1 time(s)
# greet called 2 time(s)
# 2

The wraps(func)(self) call updates the instance’s __dict__. Note that WRAPPER_UPDATES is ('__dict__',). That is the standard idiom, and it works because wraps is a partial application of functools.update_wrapper. Some codebases use functools.update_wrapper(self, func) directly for the same effect.

For the class-based form to be useful, the class needs an __init__ that captures state, and __call__ that uses it. The __call__ protocol is documented at /reference/dunder-methods/dunder-call/ and the callable() builtin at /reference/built-in-functions/callable/.

The functools family

functools ships a small library of decorators that solve specific problems. The ones you will reach for most:

  • functools.wraps / update_wrapper, which copy metadata. Covered above.
  • functools.lru_cache(maxsize=128, typed=False), which memoizes with an LRU eviction policy. Exposes .cache_info() returning a CacheInfo(hits, misses, maxsize, currsize) named tuple, and .cache_clear() to reset.
  • functools.cache (3.9+), same as lru_cache(maxsize=None). Smaller and faster; never evicts. Use it when input cardinality is bounded.
  • functools.cached_property (3.8+), which turns a method into a property that runs once and caches the result on the instance.
  • functools.total_ordering, which fills in the rest of the ordering methods given __eq__ and one of __lt__, __le__, __gt__, __ge__.
  • functools.singledispatch, generic functions keyed on the runtime type of the first positional argument. Overrides register with @base.register(Type) (parens-less form is 3.7+).
  • functools.partial, which binds some positional or keyword arguments ahead of time. The argument-forwarding form partial(func, *args, **kwargs) is the same as lambda *a, **kw: func(*args, *a, **kwargs, **kw).
from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

print(fib(100), fib.cache_info())
# output: 354224848179261915075  CacheInfo(hits=98, misses=101, maxsize=None, currsize=101)

lru_cache is the workhorse here. If you want a fuller treatment of caching patterns, see /guides/lru-cache-guide/.

Method decorators are descriptors

The built-in @property, @staticmethod, and @classmethod are not magic; they are descriptors. When you decorate a method, the name on the class no longer points at a function. It points at a descriptor instance, and the descriptor protocol runs on every attribute access.

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    @property
    def magnitude(self):
        return (self.x ** 2 + self.y ** 2) ** 0.5

print(type(vars(Point)["magnitude"]).__name__)  # property
print(Point.magnitude.__get__)                 # <slot wrapper '__get__' of 'property' objects>

p = Point(3, 4)
print(p.magnitude)

# output:
# property
# <slot wrapper '__get__' of 'property' objects>
# 5.0

Three things to internalize:

  • @property returns a property instance whose __get__ (and optional __set__ / __delete__) the descriptor protocol calls for you. The full mechanics are in /guides/descriptor-protocol/ and the dunder at /reference/dunder-methods/dunder-get/.
  • @staticmethod returns a staticmethod instance. A.f(1) and A().f(1) both work and the class is not passed in.
  • @classmethod returns a classmethod instance. A.g(1) and A().g(1) both work, and the class is passed in as the first argument. A.g(1) returns (A, 1).

The reference pages: /reference/built-in-functions/property/, /reference/built-in-functions/staticmethod/, /reference/built-in-functions/classmethod/. When you write a custom descriptor for @property-style behaviour, hook __set_name__ so the descriptor knows its own attribute name (see /reference/dunder-methods/dunder-set-name/).

Type-hinting decorators

A decorator that drops the wrapped function’s signature and return type is a hazard. Before 3.10 you had to choose between Callable[..., Any] (loses everything) and TypeVar tricks that did not compose. typing.ParamSpec and typing.Concatenate fix that.

from typing import Callable, ParamSpec, TypeVar
from functools import wraps

P = ParamSpec("P")
R = TypeVar("R")

def logged(func: Callable[P, R]) -> Callable[P, R]:
    @wraps(func)
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        print(f"calling {func.__name__}")
        return func(*args, **kwargs)
    return wrapper

A decorator factory needs Concatenate whenever it binds a new first argument at the call site. The classic case is a method decorator that needs self or cls, but it also covers a wrapper that injects a logger, a database connection, or a retry context before the user’s real arguments show up:

from typing import Concatenate

def with_retry(max_attempts: int) -> Callable[[Callable[P, R]], Callable[P, R]]:
    def decorator(func: Callable[P, R]) -> Callable[P, R]:
        @wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            for _ in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception:
                    continue
            raise RuntimeError("exhausted retries")
        return wrapper
    return decorator

For a method decorator that captures self, swap P for Concatenate[Self, P]. typing.Self is 3.11+; on 3.10, type the receiver as the concrete class or a TypeVar bound to it. The generics side of this story is in /guides/python-typing-generics/.

Common gotchas and how to dodge them

A short list of the decorator pitfalls that bite in real code:

  • Late binding in loops. Capture with a default argument, or build decorators with a factory that takes the parameter explicitly.
  • wraps does not copy __signature__. It sets __wrapped__ so inspect.signature can follow the chain. Make sure every layer of a stacked decorator uses @wraps, or the chain breaks at the layer that forgot.
  • cached_property + __slots__. The cached value lives on the instance __dict__, so if you declare __slots__ without __dict__ in it, the lookup fails at runtime. Add __dict__ to the slots, or skip cached_property and use a plain attribute.
  • lru_cache on instance methods. self is part of the cache key, so the cache holds the instance alive forever. Either make the method a @staticmethod and cache only the real arguments, or use a per-instance cache (e.g. cachetools.func.ttl_cache keyed on the real args) so the instance can be garbage-collected.
  • Dataclass + decorator ordering. A decorator that wraps __init__ after @dataclass(frozen=True) runs against the dataclass’s overridden __setattr__ / __delattr__, and the two will fight. Put the dataclass decorator closest to the class so it runs last during class creation, then layer your wrapper outside it.
  • @contextlib.contextmanager and exceptions. The body after yield runs on __exit__; if the with block raised, the exception is throw()n into the generator at the yield point. Use try/finally for cleanup, never return value from the generator (use yield value). The contextmanager API is at /reference/modules/contextlib-module/ and the with keyword at /reference/keywords/with-keyword/.
  • Async wrappers. functools.wraps works on async def wrappers, but the wrapper itself is a coroutine function, so call it with await, not just (). Use inspect.iscoroutinefunction to test for it, not a bare callable check. The async and await keywords are at /reference/keywords/async-keyword/ and /reference/keywords/await-keyword/.

For the bigger picture on how decorators compare to __init_subclass__ and metaclasses, see /guides/python-metaclasses/. For closure mechanics that make decorators possible, see /guides/python-closures/. For inspect.signature and the __wrapped__ chain, see /reference/modules/inspect-module/.

Conclusion

A decorator is a callable that takes a callable and returns a callable. Everything else (wraps, factories, class-based decorators, the descriptor trick that powers @property) is mechanism around that core. The most useful habits are mechanical: copy metadata with wraps, capture loop variables with default arguments, type your wrappers with ParamSpec, and remember the __wrapped__ chain when something downstream complains about a missing signature.

If you only have time for one rule, internalize the two-phase model. Decoration happens at import time, the wrapper runs at call time, and the two never overlap. Once that mental model is solid, every other decorator pattern is just a different way of arranging the same three closures.

See Also