Python's last 3.11 alpha includes a deliberately small TOML reader

Reading a Python project's configuration can itself require installing a Python package. TOML's role in packaging makes that a bootstrapping problem for tools that must inspect configuration before they can build anything. Python 3.11's last planned alpha includes a standard-library reader that addresses this dependency. The design deliberately stops at parsing.

The April 5 announcement for Python 3.11.0a7 lists PEP 680 and tomllib among the changes available for testing. This remains a developer preview whose use in production is discouraged. Its significance is that the proposed interface now exists in a concrete interpreter release, so tool maintainers can evaluate its behavior and boundaries.

I find Python's packaging complexity frustrating. A standard reader for a format used by the packaging system is a useful reduction in dependencies, even though it cannot resolve the wider collection of packaging decisions. The scope here is specific enough to evaluate without treating one new module as a redesign of the ecosystem.

A bootstrapping dependency

PEP 680 identifies Python packaging as the main motivation. PEPs 517, 518 and 621 establish TOML's role, while build tools otherwise have to vendor a parser or arrange another workaround. Including a reader in the interpreter gives those tools a parser they can use before third-party installation.

The proposal bases the implementation on Tomli. Its rationale favors a small pure-Python implementation with an established test suite and maintainers willing to support inclusion. A parser does not need a C implementation merely because its consumers include build tools. The proposal says TOML parsing is rarely an application bottleneck and leaves users with unusual performance needs free to choose another library.

This boundary also explains why the addition does not automatically justify standardizing every configuration format. TOML already occupies a specified role in Python's own packaging conventions. The dependency problem follows from that existing role. Supporting a format because the toolchain already requires it is a more concrete argument than accumulating parsers for every format an application might use.

The file interface owns decoding

The module documentation at the alpha tag specifies two entry points. load reads a binary file object. loads parses a Python string. Both return a dictionary and raise TOMLDecodeError for invalid TOML.

import tomllib

with open("pyproject.toml", "rb") as configuration:
    project = tomllib.load(configuration)

The binary mode is intentional. PEP 680 explains that the parser must apply UTF-8 decoding, independent of the platform's default text encoding. It must also avoid universal-newline processing turning invalid carriage-return input into something that appears valid. Opening the file as text would move those decisions outside the parser.

The string interface has a different job. A str already represents the document as Unicode, so it does not need the same byte-decoding step. The distinction is consistent with the objects each entry point consumes. A file's encoding and a document's character content are related questions, but they are not interchangeable inputs.

The result uses ordinary Python values. Tables become dictionaries and arrays become lists. TOML's date and time forms map to the corresponding datetime types, with timezone information on offset date-times. Consumers therefore receive parsed values without taking a dependency on a separate document model.

Why there is no writer

The absence of dump and dumps follows from the use case. Build tools need to read configuration, while changing a human-maintained file introduces requirements that a plain dictionary does not express. Comments and formatting can matter to the person who owns the file even when neither affects its parsed meaning.

A parser returning ordinary values cannot preserve all that information. A comment beside a setting does not have an obvious place in a dictionary containing only the setting's semantic value. Editing a document while preserving its style consequently requires additional metadata and decisions about how edits affect the surrounding text.

PEP 680 also identifies choices that arise even when creating a new file. A writer needs formatting rules and policies for custom values. Standard-library inclusion would commit Python to an interface for those choices. The proposal leaves writing to external libraries instead of committing to an API that the motivating use cases do not require.

I support the smaller interface. Parsing configuration and editing configuration have different contracts, and the standard library can meet the first without pretending to meet the second. Applications that need to retain comments should choose a library built around that document requirement.

Numeric values remain configurable

The parser defaults to Python float for TOML floating-point values, but accepts a parse_float callable. That callable receives the original numeric string. It can produce a decimal.Decimal when the application requires a decimal representation, avoiding an intermediate conversion through a binary float.

from decimal import Decimal
import tomllib

settings = tomllib.loads("rate = 0.1", parse_float=Decimal)
assert settings["rate"] == Decimal("0.1")

The alpha documentation excludes dictionaries and lists as callback results and specifies a ValueError if the callback returns them. The hook customizes a numeric value without changing the parser's structural interpretation. This gives consumers a targeted extension point while retaining the ordinary dictionary-and-list representation for the document around it.