March 8, 2023·4 min read·engineeringpythoncpythoncompatibility

Python 3.12 alpha 6 exposes the cost of old interfaces

The March preview makes compatibility work concrete through Unicode C API removals, distutils removal and more visible string escape warnings.

Removing a deprecated interface is easy to describe in a release note and harder to absorb across a library ecosystem. Python 3.12 alpha 6 makes that distinction visible. Its changes include removing old Unicode representation fields and the standard-library distutils package. Both affect code below the application imports that most users inspect first.

The March 8 release announcement identifies this as an early developer preview. The what is new document in the alpha 6 source tag records the implementation state available for investigation. Its introductory note says the document itself remains a draft. This is a useful point to distinguish changes already present in the preview from assumptions about the eventual final release.

I value Python chiefly because its extension ecosystem makes difficult numerical work accessible. That dependence makes early compatibility information useful even when an application has little interest in a new interpreter feature. An import failure in generated C code can prevent access to the library that justified using Python.

Unicode cleanup crosses the extension boundary

The alpha documentation records the removal of wstr and wstr_length from Unicode objects under PEP 623. It reports a reduction of eight or sixteen bytes per Unicode object on 64-bit platforms. Removing representation state therefore has a cost and a benefit at different levels. Objects become smaller, while extensions using the old representation must change.

The C API porting section lists removed legacy interfaces such as PyUnicode_AsUnicode and PyUnicode_FromUnicode. It directs extension authors toward APIs based on UTF-8 or wchar_t. Argument parsing formats that depended on Py_UNICODE*, including u and Z, also disappear. The relevant repair depends on the representation that the consuming native code actually requires.

This is not the same as changing the meaning of an ordinary Python string. Application code using str need not know the object fields. Native code that accesses a historical representation has a more specific dependency, and that dependency can survive inside generated extension sources even when nobody recently wrote it by hand.

The release announcement separately flags an internal integer representation change as a possible problem for Cython-generated code. That warning is useful because it identifies a class of dependency without suggesting that every integer operation changes for Python users. Generated code still participates in the interpreter compatibility contract. Regenerating or updating it may be part of adopting the new interpreter.

An alpha can expose these failures while both interpreter work and dependent package work are in progress. The useful report identifies the removed interface or changed assumption. Reporting only that an application cannot install loses the distinction between an unavailable distribution artifact and source code that cannot compile against the interpreter.

Distutils removal changes where build support comes from

The preview removes distutils from the standard library. The announcement points out that setuptools continues to provide the module. A build that succeeds in one environment and fails in another can therefore depend on whether setuptools supplies the compatibility implementation.

That distinction matters for interpreting successful tests. An environment with build tools already installed may hide an assumption that a fresh interpreter no longer satisfies. The source of the module has changed even if an existing import continues to work. Understanding which package supplies it is part of understanding the build dependency.

I find this aspect of Python packaging frustrating. The same import spelling can conceal different providers and installation histories. The March preview at least identifies the boundary explicitly, so build tooling can declare the dependency it requires instead of relying on an interpreter component scheduled for removal.

The alpha source document also says CPython itself no longer uses setup.py to build its shared C extension modules. Its configure script detects headers and libraries, and Makefile rules build the extensions. This is a change to building CPython. It should not be confused with a statement that every third-party project must use the interpreter build system.

String warnings distinguish two parsers

Invalid backslash escape sequences now produce SyntaxWarning instead of DeprecationWarning. The alpha documentation uses a regular expression as its example. A regular-expression escape and a Python string escape belong to different parsers, so a spelling meaningful to the regular-expression engine can still be invalid in the enclosing string literal.

The documented form uses a raw string for the pattern. That leaves backslashes available for the regular-expression parser without asking the Python string parser to interpret them as its own escapes. The warning makes a previously less visible source-level problem easier to identify.

This change is especially relevant to code that generates patterns or embeds them among other string literals. The required correction depends on the intended string value. Adding a raw-string prefix mechanically to every affected literal would ignore cases where Python escapes were intentional alongside regular-expression escapes.

The preview also improves diagnosis

Alpha 6 includes improved exception suggestions. The tagged documentation shows a missing standard-library import producing a suggestion to import sys, and an unresolved instance attribute name suggesting self.blech. Those messages help identify a likely mistake while retaining the exception itself.

Linux perf integration supplies another diagnostic improvement already recorded in the preview. The alpha documentation names PYTHONPERFSUPPORT and -X perf as ways to activate support for reporting Python function names in profiles. That makes the interpreter visible to a system profiling workflow that also observes native execution. For applications using extension libraries, relating Python coordination to native work can explain behavior that either view alone leaves incomplete.