June 27, 2018·6 min read·engineeringpythondataclassescontextvars

Python 3.7 makes record behaviour and asynchronous context explicit

Data classes generate ordinary object methods, while context variables associate state with execution context. Python 3.7 also formalizes dictionary order and introduces opt-in postponed annotations.

Thread-local state assumes that the thread identifies the work currently running. Asynchronous execution weakens that assumption because separate tasks can share a thread. A library that stores the current request or another contextual value must then distinguish the tasks using it. Python 3.7 supplies a standard mechanism for making that distinction.

Python 3.7.0 is released on June 27. Context variables are one of its new library modules. Data classes are another, addressing a separate form of repetition in ordinary object definitions. Both changes let programmers state an intention through a standard mechanism, while preserving familiar Python operations at the point where the application uses the result.

I appreciate Python's protocol-based object model more than its surrounding packaging arrangements. These additions fit that preference. They give libraries standard ways to express object behaviour and contextual state, reducing the amount of private machinery an application must understand to use them correctly.

Context belongs to the running work

The contextvars documentation shipped in 3.7.0 recommends context variables for stateful context managers to prevent state from leaking into other concurrent code. ContextVar declares a variable whose value depends on the current context. The module also provides Context and copy_context() for frameworks managing execution.

A variable can have a default. Without a value in the current context, get() uses an explicitly supplied lookup default, then the variable's own default if available. Otherwise it raises LookupError. That last case is useful when absence is an error in the program's execution arrangement. A library need not assign an arbitrary sentinel value and treat it as ordinary application data.

from contextvars import ContextVar

request_name = ContextVar("request_name", default="unassigned")

def current_name():
    return request_name.get()

token = request_name.set("inventory")
try:
    assert current_name() == "inventory"
finally:
    request_name.reset(token)

assert current_name() == "unassigned"

The token represents the previous binding, including the possibility that no binding existed. Resetting with it restores that state. The finally block expresses the lifetime of the temporary value, so an exception in the intervening operation does not leave this binding active in the current context.

This example is deliberately synchronous so that the binding and restoration are visible without introducing a server. The 3.7 documentation also provides an asynchronous server example, where a context variable makes the client address available to code called inside the task handling that client. It states that asyncio supports context variables natively without extra configuration.

The practical gain concerns library interfaces. A function that needs contextual information can retrieve it without requiring every intermediate caller to accept and forward an additional parameter. Explicit parameters remain useful when the information is part of the function's actual input. Context variables address the different case in which execution already establishes a surrounding context that several libraries need to consult.

Copying a context preserves a separate set of bindings

copy_context() returns a copy of the current context in constant time, according to the release documentation. Context.run() executes a callable with that context current. Changes to context variables during the call remain in the context used for the call, while the surrounding context retains its own bindings.

The copy is shallow. That qualification distinguishes a separate mapping of bindings from a recursive copy of application objects. Placing a mutable object in a context variable does not imply that copying the context duplicates everything reachable through the value. The mechanism controls contextual bindings, and the application still decides what mutation of its values means.

The documentation also restricts how a context can be entered. Re-entering the same context recursively or using it simultaneously from more than one operating-system thread raises RuntimeError. Framework code therefore manages context ownership as well as obtaining a context object. A copied mapping does not grant unrestricted simultaneous use of one execution context.

Context variables should be created at module level, rather than inside closures, because context objects hold strong references to them. That rule concerns the variable objects themselves. Request-specific data belongs in their contextual values; repeatedly constructing the declarations inside request handlers would give those objects a different lifetime from the intended module-level declarations.

Data classes generate methods from field declarations

The 3.7.0 dataclasses reference describes a decorator that reads annotated class variables as fields. It returns the same class after adding appropriate special methods. The default options generate initialization, representation and equality, while ordering is disabled by default.

from dataclasses import dataclass, field
from typing import List

@dataclass
class Batch:
    name: str
    items: List[str] = field(default_factory=list)

first = Batch("incoming")
second = Batch("incoming")
first.items.append("part")
assert second.items == []

The factory creates a new list when an instance needs its default. Using one mutable list as a shared default would express different ownership. The documentation explains that dataclasses reject defaults of the known mutable built-in types list, dict and set, while a default factory provides a way to construct a fresh value.

Field order follows the class definition and determines the generated methods' field order. Equality compares the instances as tuples of their fields, requiring identical types. That type requirement matters when different records happen to contain similar values. The generated operation follows the class's declared record structure instead of treating matching positions alone as enough to establish equality across unrelated classes.

Annotations identify fields, but ordinary field annotations do not impose runtime type validation. The reference describes special handling for class variables and initialization-only variables. The general design otherwise leaves the annotation's meaning to typing tools and application code. An annotated List[str] does not turn the generated constructor into an input-validation service.

I favour this use of generation when the record semantics are actually appropriate. Repeating attribute assignments and equality code creates maintenance work without adding domain information. A class whose identity or invariants require custom behaviour still needs that behaviour expressed explicitly.

Mutability affects hashing and replacement

The decorator's hashing rules reflect the interaction between equality and mutation. With equality enabled and freezing disabled, the default result is unhashable. With equality and freezing both enabled, dataclasses generate a hash by default. The reference explains that frozen=True adds assignment and deletion methods that raise FrozenInstanceError, emulating read-only instances.

Freezing therefore describes an object's assignment behaviour, not a proof that all referenced values are immutable. A record author still needs to account for the values stored in its fields. This is particularly relevant when choosing whether an object belongs in a dictionary or set, because the generated operations inherit the meaning of those field values.

The replace() helper creates another instance through the class's initialization method. Consequently, it also runs __post_init__ when the generated initializer calls that method. Fields excluded from initialization are not simply copied over. The documentation explains that they are initialized again through the class's initialization logic, which makes replacement different from copying every attribute mechanically.

This behaviour is useful for derived fields. If one field determines another during initialization, constructing a replacement should apply that relationship to the new input. The class author must decide whether the generated initialization and replacement semantics match the actual record, particularly when some fields are not constructor arguments.

The reference also supplies fields() for examining a record's declared fields and asdict() for converting an instance to a dictionary. The conversion descends into nested dataclasses and containers such as lists and tuples. That provides a standard route for code that needs a structural representation of a record. It does not eliminate the need to define an external data contract. A field's Python name and its role in generated equality do not establish what an external consumer should receive. A library can use the helpers while still choosing a separate representation for data exchanged with another system.

Order and annotations have different compatibility rules

The release's what's new document distinguishes a language guarantee from an opt-in evaluation change. Dictionary insertion order is now an official part of the Python language specification. Code relying on that ordering can state its requirement at the language-version level, instead of treating it solely as a particular implementation's behaviour.

Postponed annotation evaluation, specified by PEP 563, requires from __future__ import annotations in each module that requests it. The compiler stores annotation expressions as strings, allowing forward references and avoiding their immediate evaluation. Runtime consumers can resolve them with typing.get_type_hints() when needed.

The distinction affects libraries inspecting declarations. Dictionary ordering is a guarantee the release makes generally. String-based annotation storage depends on the producing module's future import. A consumer of annotations must therefore account for the producer's evaluation policy, while a consumer of dictionary iteration can rely on the insertion-order guarantee supplied by Python 3.7.