Python Decorators Deep Dive: From Closures to ParamSpec
This article picks up where the introductory /guides/python-decorators/ leaves off. If you have not written a wrapper function or used functools.wraps before, read that one first. Here, you will see what actually happens at the byte-code level when the interpreter hits @decorator, why the late-binding loop trap exists, and how to type-hint a parameterized decorator so mypy and your editor stay useful.
All code targets Python 3.10+. The ParamSpec section needs 3.10+; the functools.cache call needs 3.9+; functools.cached_property needs 3.8+.
Two-phase execution: what runs at import time vs. call time
A decorator is just a callable that takes a function and returns a function. The interpreter applies it in two distinct phases, and confusing the two is the source of half the decorator bugs in the wild.
Phase one runs at definition time, the moment the def statement finishes. The interpreter binds the function object, evaluates the decorator expression, calls it with the function, and binds the result to the original name. Phase two runs every time someone calls the decorated name.
def shout(func):
print(f"decorating {func.__name__}")
return func
@shout
def hello():
return "hi"
# output:
# decorating hello
The print fires once, at import time, even though hello is never called. That is why decorators can do expensive work (caching, registering, building wrappers) up front without paying for it on each invocation. It is also why side effects in a decorator run during module import, which matters in tests, REPL sessions, and any framework that introspects the module before you expect.
Closures are the engine (and the late-binding trap)
A typical decorator closes over the wrapped function. Python captures the binding by reference, not by value, and that one fact is responsible for the most common decorator bug: the late-binding loop trap.
# BAD: every lambda returns the same value
funcs = [lambda: i for i in range(3)]
print([f() for f in funcs])
# output: [2, 2, 2]
The body of each lambda looks up i from the enclosing scope at call time, after the loop has finished and i has settled on its last value. The same trap shows up when you build a list of decorators in a loop:
def make_multiplier(n):
return lambda x: x * n
# BAD: all multipliers are 4
multipliers = [make_multiplier(i) for i in range(3)]
print([m(10) for m in multipliers])
# output: [40, 40, 40]
The fix is to capture the value as a default argument, which binds at definition time, not at call time. Default arguments are evaluated exactly once when the function object is created, so each lambda in the list ends up holding its own private value of i rather than a shared reference to the loop variable:
funcs = [lambda i=i: i for i in range(3)]
print([f() for f in funcs])
# output: [0, 1, 2]
If you are building decorators in a loop, the same pattern applies. The closure captures the loop variable by reference, not by value. For more on how closures store state across calls, see /guides/python-closures/.
functools.wraps in depth
functools.wraps is a thin convenience over functools.update_wrapper. The two constants it uses are public and worth knowing:
WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__annotations__', '__doc__'), copied onto the wrapper by direct assignment.WRAPPER_UPDATES = ('__dict__',), merged in place so the wrapper keeps its own extra attributes.
wraps does not copy __signature__. Instead, it sets __wrapped__ on the wrapper, and inspect.signature follows that chain by default (follow_wrapped=True since 3.5). That is why stacking decorated functions still produces the right signature, as long as every layer uses @wraps.
from functools import wraps
import inspect
def logged(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(f"calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
@logged
@logged
def add(a: int, b: int) -> int:
return a + b
print(add.__name__) # add
print(inspect.signature(add)) # (a: int, b: int) -> int
# output:
# add
# (a: int, b: int) -> int
Without wraps, add.__name__ would be wrapper and the signature would be (*args, **kwargs). The whole functools module is documented at /reference/modules/functools-module/.
Decorator factories: decorators that take arguments
A decorator factory is a function that returns a decorator. It is how you parameterize behavior: repeat(n=3) instead of repeat. The structure has three nested callables:
- Outer factory, which receives the parameter (
n=3) and builds a closure over it. - Middle decorator, which receives the function and builds a closure over it.
- Inner wrapper, which runs at call time and has access to all of the above.
from functools import wraps
def repeat(n=3):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
return [func(*args, **kwargs) for _ in range(n)]
return wrapper
return decorator
@repeat(n=4)
def roll():
import random
return random.randint(1, 6)
print(roll())
Notice the call site: @repeat(n=4) calls repeat once (returning decorator), which is then applied to roll. Compare that to @logged where there is no parameter. The decorator itself is the only callable in the chain, so the call site just writes the decorator name directly above the function.
A clean way to read a factory is “the user is configuring a wrapper.” Two layers of closure look like overkill for one parameter, but the same shape scales to logging levels, retry counts, cache backends, feature flags, and so on. The factories pattern shows up everywhere in real codebases.
Class-based decorators: stateful wrappers
A class with __call__ is a callable, so it is a decorator. The class form shines when the decorator needs to keep state, such as a call counter, a cache, or a list of registered handlers.
from functools import wraps
class CountCalls:
def __init__(self, func):
wraps(func)(self) # copy metadata onto the instance
self.func = func
self.calls = 0
def __call__(self, *args, **kwargs):
self.calls += 1
print(f"{self.func.__name__} called {self.calls} time(s)")
return self.func(*args, **kwargs)
@CountCalls
def greet(name):
return f"hello {name}"
greet("alice")
greet("bob")
print(greet.calls)
# output:
# greet called 1 time(s)
# greet called 2 time(s)
# 2
The wraps(func)(self) call updates the instance’s __dict__. Note that WRAPPER_UPDATES is ('__dict__',). That is the standard idiom, and it works because wraps is a partial application of functools.update_wrapper. Some codebases use functools.update_wrapper(self, func) directly for the same effect.
For the class-based form to be useful, the class needs an __init__ that captures state, and __call__ that uses it. The __call__ protocol is documented at /reference/dunder-methods/dunder-call/ and the callable() builtin at /reference/built-in-functions/callable/.
The functools family
functools ships a small library of decorators that solve specific problems. The ones you will reach for most:
functools.wraps/update_wrapper, which copy metadata. Covered above.functools.lru_cache(maxsize=128, typed=False), which memoizes with an LRU eviction policy. Exposes.cache_info()returning aCacheInfo(hits, misses, maxsize, currsize)named tuple, and.cache_clear()to reset.functools.cache(3.9+), same aslru_cache(maxsize=None). Smaller and faster; never evicts. Use it when input cardinality is bounded.functools.cached_property(3.8+), which turns a method into a property that runs once and caches the result on the instance.functools.total_ordering, which fills in the rest of the ordering methods given__eq__and one of__lt__,__le__,__gt__,__ge__.functools.singledispatch, generic functions keyed on the runtime type of the first positional argument. Overrides register with@base.register(Type)(parens-less form is 3.7+).functools.partial, which binds some positional or keyword arguments ahead of time. The argument-forwarding formpartial(func, *args, **kwargs)is the same aslambda *a, **kw: func(*args, *a, **kwargs, **kw).
from functools import lru_cache
@lru_cache(maxsize=None)
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(100), fib.cache_info())
# output: 354224848179261915075 CacheInfo(hits=98, misses=101, maxsize=None, currsize=101)
lru_cache is the workhorse here. If you want a fuller treatment of caching patterns, see /guides/lru-cache-guide/.
Method decorators are descriptors
The built-in @property, @staticmethod, and @classmethod are not magic; they are descriptors. When you decorate a method, the name on the class no longer points at a function. It points at a descriptor instance, and the descriptor protocol runs on every attribute access.
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
@property
def magnitude(self):
return (self.x ** 2 + self.y ** 2) ** 0.5
print(type(vars(Point)["magnitude"]).__name__) # property
print(Point.magnitude.__get__) # <slot wrapper '__get__' of 'property' objects>
p = Point(3, 4)
print(p.magnitude)
# output:
# property
# <slot wrapper '__get__' of 'property' objects>
# 5.0
Three things to internalize:
@propertyreturns apropertyinstance whose__get__(and optional__set__/__delete__) the descriptor protocol calls for you. The full mechanics are in /guides/descriptor-protocol/ and the dunder at /reference/dunder-methods/dunder-get/.@staticmethodreturns astaticmethodinstance.A.f(1)andA().f(1)both work and the class is not passed in.@classmethodreturns aclassmethodinstance.A.g(1)andA().g(1)both work, and the class is passed in as the first argument.A.g(1)returns(A, 1).
The reference pages: /reference/built-in-functions/property/, /reference/built-in-functions/staticmethod/, /reference/built-in-functions/classmethod/. When you write a custom descriptor for @property-style behaviour, hook __set_name__ so the descriptor knows its own attribute name (see /reference/dunder-methods/dunder-set-name/).
Type-hinting decorators
A decorator that drops the wrapped function’s signature and return type is a hazard. Before 3.10 you had to choose between Callable[..., Any] (loses everything) and TypeVar tricks that did not compose. typing.ParamSpec and typing.Concatenate fix that.
from typing import Callable, ParamSpec, TypeVar
from functools import wraps
P = ParamSpec("P")
R = TypeVar("R")
def logged(func: Callable[P, R]) -> Callable[P, R]:
@wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
print(f"calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
A decorator factory needs Concatenate whenever it binds a new first argument at the call site. The classic case is a method decorator that needs self or cls, but it also covers a wrapper that injects a logger, a database connection, or a retry context before the user’s real arguments show up:
from typing import Concatenate
def with_retry(max_attempts: int) -> Callable[[Callable[P, R]], Callable[P, R]]:
def decorator(func: Callable[P, R]) -> Callable[P, R]:
@wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
for _ in range(max_attempts):
try:
return func(*args, **kwargs)
except Exception:
continue
raise RuntimeError("exhausted retries")
return wrapper
return decorator
For a method decorator that captures self, swap P for Concatenate[Self, P]. typing.Self is 3.11+; on 3.10, type the receiver as the concrete class or a TypeVar bound to it. The generics side of this story is in /guides/python-typing-generics/.
Common gotchas and how to dodge them
A short list of the decorator pitfalls that bite in real code:
- Late binding in loops. Capture with a default argument, or build decorators with a factory that takes the parameter explicitly.
wrapsdoes not copy__signature__. It sets__wrapped__soinspect.signaturecan follow the chain. Make sure every layer of a stacked decorator uses@wraps, or the chain breaks at the layer that forgot.cached_property+__slots__. The cached value lives on the instance__dict__, so if you declare__slots__without__dict__in it, the lookup fails at runtime. Add__dict__to the slots, or skipcached_propertyand use a plain attribute.lru_cacheon instance methods.selfis part of the cache key, so the cache holds the instance alive forever. Either make the method a@staticmethodand cache only the real arguments, or use a per-instance cache (e.g.cachetools.func.ttl_cachekeyed on the real args) so the instance can be garbage-collected.- Dataclass + decorator ordering. A decorator that wraps
__init__after@dataclass(frozen=True)runs against the dataclass’s overridden__setattr__/__delattr__, and the two will fight. Put the dataclass decorator closest to the class so it runs last during class creation, then layer your wrapper outside it. @contextlib.contextmanagerand exceptions. The body afteryieldruns on__exit__; if thewithblock raised, the exception isthrow()n into the generator at theyieldpoint. Usetry/finallyfor cleanup, neverreturn valuefrom the generator (useyield value). ThecontextmanagerAPI is at /reference/modules/contextlib-module/ and thewithkeyword at /reference/keywords/with-keyword/.- Async wrappers.
functools.wrapsworks onasync defwrappers, but the wrapper itself is a coroutine function, so call it withawait, not just(). Useinspect.iscoroutinefunctionto test for it, not a barecallablecheck. Theasyncandawaitkeywords are at /reference/keywords/async-keyword/ and /reference/keywords/await-keyword/.
For the bigger picture on how decorators compare to __init_subclass__ and metaclasses, see /guides/python-metaclasses/. For closure mechanics that make decorators possible, see /guides/python-closures/. For inspect.signature and the __wrapped__ chain, see /reference/modules/inspect-module/.
Conclusion
A decorator is a callable that takes a callable and returns a callable. Everything else (wraps, factories, class-based decorators, the descriptor trick that powers @property) is mechanism around that core. The most useful habits are mechanical: copy metadata with wraps, capture loop variables with default arguments, type your wrappers with ParamSpec, and remember the __wrapped__ chain when something downstream complains about a missing signature.
If you only have time for one rule, internalize the two-phase model. Decoration happens at import time, the wrapper runs at call time, and the two never overlap. Once that mental model is solid, every other decorator pattern is just a different way of arranging the same three closures.
See Also
- /reference/modules/functools-module/
- /reference/dunder-methods/dunder-call/
- /reference/dunder-methods/dunder-get/
- /guides/python-decorators/, the prerequisite intro
- /guides/descriptor-protocol/, the mechanism behind
@property - /guides/python-typing-generics/, covering
ParamSpecandConcatenatein depth