October 2, 2023·6 min read·engineeringpythontypinggenericscpython

Python 3.12 gives type parameters a local scope

PEP 695 reduces generic declaration boilerplate by changing where type parameters live and when alias values are evaluated. Python 3.12 also removes special parsing restrictions from f-strings.

A generic function needs a name for the type relationship it describes. Older Python typing syntax commonly introduces that name separately from the function, leaving readers to connect a module-level type variable with the declaration that uses it. Python 3.12 can put the parameter directly in the function declaration. The shorter spelling follows a more consequential change to scope.

The October 2 release includes PEP 695 type parameter syntax and the type statement. Its release-tag documentation describes unclear type-parameter scope and verbose declarations as problems with the earlier arrangement. The new syntax gives generic declarations their own parameters and supports lazy evaluation for aliases, bounds and constraints.

I prefer declarations whose relationships are visible where they are used. Python remains a language I mainly choose for its ecosystem, but this change makes a typed API easier to read without first finding a separate type-variable assignment elsewhere in the module.

The declaration owns its type parameter

A generic function can now declare its parameter between the function name and its argument list. This example returns one of the values supplied by its caller and expresses that relationship through T.

def first[T](items: list[T]) -> T:
    return items[0]

The parameter is visible within the declaration and relevant nested scopes. It does not become a module-level name after the function definition. A generic class can similarly declare class Box[T], with T available to annotations on its methods. The notation associates the parameter with the declaration that introduces it.

This matters when several declarations use the same conventional letter. Their parameters need not imply one shared module-level typing object merely because their authors chose T. Each declaration establishes its own scope. A reader can understand the relationship locally instead of inferring which separate assignments were intended to be reused.

Moving the parameter list also helps distinguish the definition of a generic from a later specialization. The declaration introduces a relationship that its callers can instantiate with different types. The brackets belong to the declaration in this case, which gives the reader a direct syntactic signal about where that relationship begins.

The syntax also accommodates parameter specifications and variadic type parameters. The release documentation shows **P for a parameter specification and *Ts for a type-variable tuple. These forms describe different relationships. A function signature and a sequence of type arguments are not interchangeable just because both can have variable length.

Bounds and constraints have distinct spellings too. A bound follows a colon, such as T: Hashable. A constrained parameter places the allowed types in a parenthesized list. The notation exposes the distinction at the declaration, where a reader needs it to understand which substitutions the generic accepts.

A type statement creates an alias object

The new alias syntax uses a soft keyword at the beginning of the statement. A generic alias can declare parameters in the same local form as a class or function.

type Pair[T] = tuple[T, T]
type Point = tuple[float, float]

The type statement creates a typing.TypeAliasType instance. Its value is evaluated on demand. That differs from treating the right side as an ordinary expression that must finish evaluating immediately when the assignment executes.

Lazy evaluation permits an alias to refer to a type defined later in the file. The name need not already resolve when the alias object is created. This is useful for mutually dependent descriptions, where requiring every referenced alias to be fully defined first would make the declaration order impossible.

The 3.12 execution-model documentation demonstrates the timing with an alias whose value divides by zero. Creating the alias succeeds. Accessing its __value__ raises the exception. The example is deliberately invalid as a useful type, but it makes the evaluation boundary observable.

Bounds and constraints introduced through the new parameter syntax follow a similar rule. The interpreter can create the type variable before evaluating its bound. Accessing the bound later performs the required evaluation. This means introspection can trigger work that declaration alone did not perform.

Code that reads alias values therefore participates in execution. It should not assume that obtaining a type object has already resolved every name or expression inside the alias. The language makes the delayed step part of the object behavior, which matters to libraries that inspect annotations and aliases at runtime.

Annotation scopes have specific rules

Supporting local parameters and delayed expressions requires a new kind of scope. Annotation scopes resemble function scopes but have several differences described in the release execution model. In particular, an annotation scope can access its enclosing class namespace.

That behavior makes class-local type descriptions possible without pretending that a method body normally resolves names through its class body. Ordinary functions defined inside a class have different lookup behavior. The new scope gives type-related declarations the access they need while preserving that existing distinction.

Generic function annotations execute within the annotation scope, but function defaults and decorators do not. Generic class bases and keyword arguments use the annotation scope, while class decorators do not. These boundaries determine which expressions can see the type parameters introduced by a declaration.

The scope also restricts some expressions. The execution model excludes yield, yield from, await and assignment expressions from expressions in an annotation scope. Inner scopes contained within it have their own rules. Type parameters cannot be rebound through a nested nonlocal statement.

These details are reasons to treat PEP 695 as a language change rather than a search-and-replace abbreviation for old declarations. Scope and evaluation timing affect runtime introspection as well as static analysis. The new syntax is useful because it expresses ownership of parameters, but tools have to understand that ownership too.

F-strings join the ordinary parser

Python 3.12 also implements PEP 701, lifting restrictions on expressions inside f-strings. An expression can reuse the enclosing quotation style and contain backslashes. Multiline expressions and comments become possible as well. The release documentation relates these changes to parsing f-strings with the PEG parser.

The practical improvement is consistency between an expression written inside an f-string and the same expression written elsewhere. A dictionary lookup no longer requires choosing its quotes around the enclosing f-string delimiter. Code generation also has fewer special restrictions to account for when constructing an embedded expression.

Error reporting benefits from the parser change. The release documentation compares a malformed expression whose older diagnostic showed an artificial expression fragment with the new diagnostic showing the full source line and a more precise location. The parser has access to the surrounding syntax needed to explain where the failure occurred.

This removes reasons to introduce temporary variables solely to satisfy a parsing restriction. It does not imply that every complicated expression belongs inside a formatted string. A local name can still explain a calculation better. The language now permits that choice to follow readability instead of a separate f-string grammar limitation.

Parallel execution has an isolation boundary

The release also includes a per-interpreter GIL through PEP 684. Subinterpreters can have distinct locks, allowing them to execute on different CPU cores. In this release the facility is available through the C API, including Py_NewInterpreterFromConfig.

That is a different arrangement from removing the GIL for threads sharing one interpreter. The lock still exists within each configured interpreter. Code embedding Python must account for isolation between them and the suitability of extensions used there. The presence of a per-interpreter lock does not make an ordinary collection of Python threads execute concurrently in one interpreter.

For me, this is another place where the library dependency matters as much as the interpreter capability. The useful question is which execution arrangement supports the objects and extensions an application needs. Python 3.12 supplies an isolation-based option at the C API while the broader optional-GIL work remains a separate project.

Packaging changes affect the environment around typed code

Distutils is removed from the standard library, and virtual environments created by venv no longer include setuptools by default. The release documentation notes that installing setuptools supplies its distutils implementation when that remains necessary. An environment created from a fresh interpreter can therefore differ from an older environment that accumulated build tools.

These changes are relevant even to a project adopting 3.12 chiefly for generic syntax. Parsing the new declarations establishes a minimum interpreter requirement for those source files. Building and installing their dependencies adds a separate compatibility requirement. A declaration can become easier to understand while its distribution still depends on explicitly providing the build packages that its toolchain uses.