Python 3.15.0a5 and testing the UTF-8 default

Python 3.15.0a5 is an extra alpha with a specific correction. The release page says that a4 accidentally used the main branch from December 23 instead of January 13. The replacement uses January 14 sources. Alpha testing includes testing the release process itself, and this is a useful example of why the exact artifact matters.

I have used Python for twenty years, usually because its libraries have already solved a substantial part of the problem. Its packaging systems remain a source of frustration for me. A new interpreter is therefore interesting partly because of what it asks of the surrounding ecosystem. Before thinking about adoption, I want to know which assumptions an application makes about its environment and whether the new release changes them.

The alpha is a developer preview, and the release announcement advises against production use. I would treat it as a separate compatibility exercise. One particularly useful exercise concerns the UTF-8 default described in PEP 686. That proposal gives a precise change to investigate without trying to evaluate every unfinished part of the next interpreter.

A default changes the meaning of omitted arguments

PEP 686 makes UTF-8 mode the default in Python 3.15. Its stated scope includes the default encoding of files, standard input and output, and pipes. This matters because a program can depend on an encoding without naming one. An omitted argument transfers the choice to the interpreter and its environment.

The proposal identifies a familiar portability problem. Developers on Unix systems often assume that reading a text file without an encoding means reading UTF-8. The historical default depends on the locale, and the assumption can fail on another machine. PEP 686 says the change mostly affects Windows users because most Unix systems already use UTF-8 locales and Python enables UTF-8 mode for the C and POSIX locales.

Changing that default makes common UTF-8 files easier to handle consistently. It also changes the behavior of programs that deliberately or accidentally rely on another encoding. The proposal explicitly warns about Unicode errors, incorrectly decoded text, and silent corruption. A successful process exit is therefore insufficient evidence that the application preserved its data.

I prefer an explicit encoding at a file boundary whose format the application owns. That is a code review decision about the data contract. The interpreter default remains relevant for existing code, dependency code, and interfaces whose callers supply the files.

The warning mechanism predates the switch

PEP 597 introduced an optional EncodingWarning in Python 3.10. It reports places where an application omits the encoding argument and uses the locale default. The warning requires an opt-in flag or environment variable. It was a preparatory mechanism for finding dependencies on an implicit encoding before changing the default.

Its motivation includes package installation. A setup script that reads a UTF-8 README without specifying its encoding may work on the author's machine and fail for a user with a different locale. Non-ASCII author names or other text can make the difference. The proposal uses that example to explain why a default can become an installation problem before a user runs the application.

The warning also helps with wrapper functions. PEP 597 provides io.text_encoding() so an API that passes an encoding argument to open() can report the warning at its caller. That is useful because the decision to omit an encoding may occur outside the function that opens the file. Searching for direct calls alone does not account for every such dependency.

The design leaves the warning optional because universally reporting omitted encodings would produce excessive noise. A program that only reads ASCII can omit the argument without encountering the portability problem described above. I would enable the warning during a deliberate compatibility run and review its results as evidence about individual file formats.

Test both intended contracts

The migration procedure in PEP 686 starts by disabling UTF-8 mode and enabling the warning. It then asks maintainers to decide whether each affected operation should use UTF-8 or the locale encoding. Finally, they test with UTF-8 mode enabled. This sequence separates identifying assumptions from checking behavior under the new default.

For a file defined to contain UTF-8, the intent can be written directly. This example makes the format assumption visible at the call site.

with open("README.md", encoding="utf-8") as source:
    description = source.read()

Where compatibility with locale-encoded input is intentional, PEP 597 provides a named option. This example preserves that choice explicitly.

with open("local-report.txt", encoding="locale") as source:
    report = source.read()

PEP 686 specifies that encoding="locale" must retain locale behavior even when UTF-8 mode is active. It records the correction of that interaction in Python 3.11. It also distinguishes locale.getencoding(), which ignores UTF-8 mode, from locale.getpreferredencoding(False), which the mode affects. Code that queries an encoding deserves the same review as code that opens a file.

For tests, I would include text that exercises the intended encoding and assert the decoded value. ASCII-only fixtures cannot distinguish many encoding assumptions. A fixture for a locale-specific workflow must represent that workflow's contract. Replacing every omitted argument with UTF-8 without examining the input would conceal the decision the migration requires.

The proposal keeps an escape option through PYTHONUTF8=0 or -X utf8=0. That supports investigation and temporary compatibility, but an application still needs to establish which encoding its input requires. For this alpha, the useful output of testing is a set of explicit decisions about those boundaries and reproducible failures that maintainers can report while the release is still under development.