Functions

Functions are objects#

def is an executable statement. It builds a function object at run time and binds it to a name in the enclosing namespace. The object carries __code__ (a compiled code object), __defaults__, __kwdefaults__, __closure__, __globals__, __annotations__ and __dict__.

Because they are ordinary objects, functions can be stored in lists, passed as arguments, returned, and given attributes. That is what makes decorators, callbacks, key= arguments and dependency injection possible without any special language machinery.

Argument binding#

The full parameter grammar, in order:

def f(pos_only, /, standard, *args, kw_only, **kwargs)

• Everything before / is positional-only (PEP 570, Python 3.8). This is how many C builtins have always behaved.
• Everything after a bare * or after *args is keyword-only (PEP 3102).
• *args collects surplus positionals into a tuple; **kwargs collects surplus keywords into a dict.

from functools import lru_cache


def f(a, b=2, *rest, key=None, **extra):
    return a, b, rest, key, extra


print(f(1))
print(f(1, 3, 4, 5, key="k", flag=True))
print("defaults live on the object:", f.__defaults__, f.__kwdefaults__)

fs = [lambda: i for i in range(3)]         # late binding: i is read at call time
print("late binding :", [g() for g in fs])
fs = [lambda i=i: i for i in range(3)]     # bound at definition time instead
print("early binding:", [g() for g in fs])

calls = 0


@lru_cache(maxsize=None)
def fib(n):
    global calls
    calls += 1
    return n if n < 2 else fib(n - 1) + fib(n - 2)


print("fib(30):", fib(30), "| bodies executed:", calls)

(1, 2, (), None, {})
(1, 3, (4, 5), 'k', {'flag': True})
defaults live on the object: (2,) {'key': None}
late binding : [2, 2, 2]
early binding: [0, 1, 2]
fib(30): 832040 | bodies executed: 31

Three results in that output are worth dwelling on.

Defaults are stored on the function object, evaluated once when def executes. That is the whole explanation of the mutable-default trap, and it is also why def f(x, t=time.time()) freezes a timestamp at import.

Late binding in closures. A closure captures the variable, not the value at capture time, via a cell object. All three lambdas share one cell holding i, which is 2 after the comprehension finishes. The i=i trick works because default values are evaluated eagerly. This is a live bug generator in code that builds a list of per-layer or per-feature callbacks in a loop.

Memoisation collapses exponential recursion. Naive fib(30) executes 2,692,537 bodies; with lru_cache it executes 31. The cache key is the argument tuple, so arguments must be hashable — a real constraint when the argument is a NumPy array or a DataFrame.

Frames, closures and cost#

Each Python-level call builds a frame holding the local variable array, the value stack, and a pointer to the caller. Before 3.11 that meant heap-allocating a frame object per call. PEP 659's work, plus the frame changes in 3.11, inlined Python-to-Python calls into the eval loop, removing most of the per-call C stack push and lazily materialising frame objects only when introspection demands one. Calls got substantially cheaper; they did not become free.

The practical rule is unchanged: a Python function called once per array element is a performance defect, and a Python function called once per batch is free. This is the same cost boundary described in loops and conditions.

Scope resolution follows LEGB — local, enclosing, global, builtin. The compiler decides local versus non-local statically, from whether the name is assigned anywhere in the body, which is what produces UnboundLocalError rather than a fall-through to the global. nonlocal (PEP 3104) rebinds in the nearest enclosing function scope; global rebinds at module scope.

Decorators#

A decorator is a function taking a function and returning a replacement. @d above def f is exactly f = d(f), evaluated at definition time.

The one obligation is preserving metadata. functools.wraps copies __name__, __doc__, __module__, __qualname__ and __wrapped__ onto the wrapper, without which every decorated function in your codebase reports itself as wrapper in tracebacks and documentation.

Decorators earn their place in ML code for cross-cutting concerns: @lru_cache for expensive pure lookups, @torch.no_grad() for inference, @functools.singledispatch for type-based dispatch, @dataclass for config objects, and timing or retry wrappers around data loading.

Purity, and why it matters here#

A pure function depends only on its arguments and mutates nothing outside itself. Purity is not an aesthetic preference in machine-learning code; it is what makes four practical things possible.

• Reproducibility. A preprocessing step that reads a global or mutates its input cannot be rerun to the same result.
• Caching. Memoisation is sound only for pure functions.
• Parallelism. multiprocessing and joblib serialise arguments across process boundaries, so hidden shared state either breaks or silently diverges.
• Testing. A pure function needs no fixtures.

The two impure things that matter most in practice are global random state and in-place mutation of arrays. Pass an explicit numpy.random.Generator rather than calling np.random.seed, and prefer returning a new array over writing through a view whose owner you cannot see.

Typing#

PEP 484 annotations on functions are the highest-value place to use them, because the signature is the contract. PEP 604 allows int | None instead of Optional[int]. typing.Callable, ParamSpec (PEP 612) and TypeVar let a decorator preserve its wrapped signature for a checker.

Annotations are not enforced at run time. pydantic does enforce them, at a real cost, which is why it appears at the edges of a system — parsing configs and API payloads — rather than in inner loops.

References#

• PEP 3102 — Keyword-Only Arguments, Talin, 2006.
• PEP 3104 — Access to Names in Outer Scopes, Yee, 2006.
• PEP 318 — Decorators for Functions and Methods, Smith, Coghlan et al., 2003.
• PEP 570 — Python Positional-Only Parameters, Hastings, Galindo et al., 2018.
• PEP 612 — Parameter Specification Variables, Lee, 2019.
• PEP 659 — Specializing Adaptive Interpreter, Shannon, 2021.
• CPython source, Objects/funcobject.c and Python/ceval.c.

What to learn next#

• NumPy — ufuncs, and functions that dispatch to compiled kernels.
• Python for machine learning — composing pure stages into a pipeline.
• Loops and conditions — the call-overhead boundary in more detail.