October 5, 2020·6 min read·engineeringpythontypingcollectionsparsing

Python 3.9 makes built-in collections generic without checking their elements

Python 3.9 removes the parallel collection names needed for type annotations. GenericAlias preserves type metadata while ordinary collection instances keep their existing runtime behaviour.

Annotating a Python list has required a second name for the same familiar collection. Code constructs a list but imports List from typing to describe its element type. Python 3.9, released today, lets built-in collections carry generic parameters directly. The resulting list[str] expression is simpler, but its runtime meaning deserves more explanation than the syntax change alone.

PEP 585 identifies the duplicated collection hierarchy as the problem. Static typing developed incrementally within the constraints of the existing runtime, producing parallel names in typing. The proposal removes the need for those names in standard collection annotations while retaining metadata that runtime tools already use. It does not make Python check the contents of a list when an element is appended.

I like the reduction in bookkeeping. Python's protocol model already lets a library participate in ordinary language operations. Requiring a separate annotation name for a built-in collection made the relationship between the value and its declaration unnecessarily awkward.

The annotation uses the collection's own name

The release's feature documentation shows built-in collection types accepting parameters in annotations. A function can describe a list of names with list[str], and a nested configuration structure can use dict[str, list[int]]. The language no longer needs an import solely to give those built-in collections generic spelling.

This is more than a change to how annotations are displayed. PEP 585 specifies __class_getitem__ support for the participating collections. Subscribing the class produces an object representing the parameterized form. That expression can consequently appear outside an annotation, including in a type alias evaluated at runtime.

The distinction matters for compatibility with older interpreters. The PEP explains that postponed annotations can permit the spelling inside annotations on Python 3.7 and later, subject to external tooling understanding it. That does not give the older interpreter the same runtime subscription behaviour. A type alias evaluated outside an annotation still needs the runtime capability.

A library supporting several Python versions therefore needs to distinguish delayed annotation evaluation from evaluating the same expression as ordinary code. Those are different execution contexts even when the source text looks identical. Replacing every imported annotation name without considering that distinction can change the minimum interpreter requirement.

GenericAlias preserves information about the declaration

The runtime result is a types.GenericAlias. PEP 585 describes it as a thin proxy for the underlying collection type, with metadata recording the origin and arguments. Its representation includes the parameters. The __origin__ attribute identifies the original class, and __args__ records the supplied type arguments.

The following example makes those relationships explicit. It uses runtime assertions rather than assuming that the printed spelling establishes how the object behaves.

from types import GenericAlias

names_type = list[str]
assert isinstance(names_type, GenericAlias)
assert names_type.__origin__ is list
assert names_type.__args__ == (str,)
assert names_type != list[int]

That retained metadata gives introspection tools something useful to inspect. The PEP names API generation and runtime checking implemented by libraries as existing uses of generic information. Retaining it preserves those possibilities while removing the duplicated names that callers previously imported.

One rejected alternative was to make subscription return the bare class, so that list[str] would become list. That would have been simpler for the runtime but would discard the parameters. It would also differ from the existing parameterized objects in typing, which retain their arguments. The chosen design preserves information without making every collection instance carry it.

The distinction between a parameterized declaration and an ordinary instance is central to this feature. The alias describes a requested type relationship. It is not a new implementation of the underlying container with a different set of storage rules.

Existing typing imports have a migration path

PEP 585 deprecates the parallel collection names in typing, but specifies that this deprecation does not generate runtime DeprecationWarning messages. Type checkers may instead warn when a project's declared target is Python 3.9 or newer. That keeps the decision tied to the version the program intends to support.

For a library that still supports an earlier interpreter, retaining the compatible spelling can therefore be deliberate. The availability of the simpler spelling does not require changing every supported-version promise at once. A checker needs the target version to distinguish unnecessary duplication from a compatibility choice.

The alias metadata also includes __parameters__, a lazily computed tuple of unique type variables found in the arguments. This supports declarations that have not yet supplied every concrete type. The PEP specifies preservation of the alias's origin and parameters through copying and pickling as well. Introspection can therefore depend on an explicit representation of the declaration, without requiring that representation to become part of each container instance.

Constructed containers still have ordinary runtime behaviour

PEP 585 permits constructing a collection through its parameterized alias. The resulting object is an ordinary collection, and its generic arguments are erased during instance creation. The interpreter does not check later operations against the alias's arguments.

values = list[str]()
assert type(values) is list
values.append(42)
assert values == [42]

This example is deliberately inconsistent with the advertised element type. Runtime Python accepts the append. A static checker can diagnose incompatible operations from the declaration, but the container itself does not become an element validator. That boundary keeps annotation metadata separate from the runtime mechanics of a list.

The PEP rejected disallowing construction through these aliases. Its discussion notes that other parameterized standard-library types can already be instantiated, and that built-in collections should not require an exceptional rule here. Allowing construction does not imply adding element checks to the constructor or to every subsequent mutation.

I want the typing boundary stated plainly. The new spelling improves declarations and gives tools consistent metadata. It does not provide the runtime guarantees of a language that enforces collection element types during execution, and application code should not assign it those guarantees.

isinstance does not reinterpret the parameters

Calling isinstance with a parameterized alias raises TypeError, as does using such an alias as the second argument to issubclass. PEP 585 considered treating list[str] like the bare list for these checks and rejected it. An expression that appears to ask whether a list contains strings would otherwise return true for a list of integers.

The alternative of performing a full element check was also outside the proposal's scope. Such checking has to handle nested types and type variables, and iterating through some collections can consume them. The syntax does not define one general runtime validation algorithm for all participating collections.

Code that intentionally wants a base-class check can inspect the alias's origin. That answers whether an object belongs to the underlying collection class. It leaves element validation as a separate operation with its own semantics, including what to do about an iterator that cannot be traversed twice.

This is a useful limitation for library authors. Runtime inspection can obtain the declared parameters, but the library must decide what evidence establishes a match. Simply handing the alias to isinstance cannot substitute for that design decision.

Dictionary operators have their own compatibility rules

The same release adds dictionary merge and update operators under PEP 584. left | right creates a new dictionary, and values from the right-hand dictionary win when keys overlap. The in-place form, |=, updates the existing dictionary.

defaults = {"workers": 2, "debug": False}
local = {"workers": 4}
combined = defaults | local
assert combined == {"workers": 4, "debug": False}
assert defaults["workers"] == 2
defaults |= [("workers", 6)]
assert defaults["workers"] == 6

The iterable of pairs in the final update is intentional. PEP 584 permits the augmented form to accept inputs supported by dict.update, while ordinary union requires dictionaries or dictionary subclasses. The operators share an override rule but have different operand contracts. Neither annotation syntax nor a generic alias changes those runtime contracts.

The example also shows why merge order belongs in the code's meaning. Reversing operands changes which value survives. A configuration merge can express a useful precedence policy, but the operator does not decide which source should have authority. Its documented right-hand override rule supplies the mechanism.

The parser changes without requiring a new annotation language

Python 3.9 also replaces its LL(1) parser with a PEG-based parser. The versioned feature documentation describes comparable performance and greater flexibility for grammar work. It states that the new parser produces the same abstract syntax tree as the old parser through ast.

For this release, the old parser remains selectable through -X oldparser or PYTHONOLDPARSER=1. Those are compatibility provisions available now. They should not be mistaken for evidence that built-in generics depend on a new parser-only syntax. PEP 585 explains the collection change through class subscription and runtime representation.

These distinctions make the practical scope of the release clearer. A list[str] alias carries inspectable arguments, constructing it produces a normal list, and element checks remain the responsibility of tools or explicit application code. The new runtime representation supports one spelling across annotations and evaluated aliases while preserving the existing container's operations.