Python 3.9.4 restores the thread-state binary layout

A change to a C structure can leave source code compilable while invalidating an existing binary. That distinction explains the urgency of Python 3.9.4, released on April 4. The release restores compatibility after Python 3.9.3 changed the layout of PyThreadState. An application need not contain a syntax error, or even changed Python code, to encounter this kind of regression.

The 3.9.4 changelog identifies the affected condition precisely. Binaries built with earlier Python 3.9.x headers could encounter an ABI break on 32-bit platforms. The hotfix reverts the earlier fix associated with issue 42500. Calling this a routine library update would obscure the reason an otherwise compatible maintenance release needed replacement.

I choose Python in large part because its extension ecosystem supplies capabilities that would otherwise require substantial separate work. That makes the interface between the interpreter and compiled code a practical concern. Source-level convenience depends on agreements implemented below the source language.

A field type changes the compiled contract

Comparing the 3.9.3 thread-state header with the 3.9.4 header shows the reverted definition. The earlier release has an integer named recursion_headroom after recursion_depth. The hotfix restores a character named overflowed in that position. Both versions then contain recursion_critical, another character, followed by stackcheck_counter, an integer.

The fields describe recursion handling, but the compatibility problem arises from their physical representation. Compilers determine field offsets and the total size of a structure according to the target ABI's layout and alignment rules. Replacing a character with an integer can change those results. Native code compiled against one declaration does not consult a new header when the resulting binary runs.

This is the mechanism described in Python's own stable ABI documentation for this release. Adding a field or changing its type can preserve API compatibility while breaking ABI compatibility. A compiler can accept the same source after a rebuild even when an already compiled module has incompatible expectations about memory.

The changelog limits its concrete report to 32-bit platforms using binaries compiled with earlier 3.9.x headers. That qualifier matters. It does not establish that every interpreter invocation or every extension fails. It identifies an architecture and artifact history that expose the changed structure. A successful run on a different architecture would not refute that report.

Rebuilding changes what the test proves

A build against the new headers can adapt native code to the new declaration. That makes compilation useful for checking source compatibility, but insufficient for detecting this particular regression. A newly compiled extension may agree with the new interpreter while an existing distributed binary still expects the earlier layout.

The relevant compatibility comparison therefore includes a previously built artifact. The changelog's reference to earlier headers describes exactly that relationship. Packaging tests that rebuild every dependency before exercising it can miss a requirement that applies to already installed native modules. The resulting application may have the same dependency names and source versions while using materially different machine code.

The hotfix resolves the immediate mismatch by restoring the earlier structure definition. It does not require downstream authors to discover and rebuild every affected binary as the intended long-term response to an incompatible maintenance update. That choice preserves the compatibility users expected within the release series.

This issue also illustrates why the interpreter version alone cannot describe a native dependency completely. The headers used at compilation and the architecture both matter. When a failure depends on those properties, reproducing it requires the original combination, rather than an installation assembled afresh on another machine.

The limited API addresses a related boundary

Python's stable ABI documentation describes Py_LIMITED_API, which hides interpreter details from an extension in exchange for a more stable binary interface. Selecting a minimum version also determines which stable-ABI functions the extension may reference. The mechanism is explicit, so code that uses unrestricted interpreter structures cannot assume it has obtained those guarantees automatically.

This is a useful design distinction even though it does not replace the hotfix. Hiding structure details reduces the number of layout decisions that become dependencies of downstream binaries. Extensions that require those details make a different tradeoff. Their implementation can depend more closely on the interpreter, and compatibility then requires greater coordination.

I prefer interfaces that make such dependencies visible. Python's extension machinery is valuable precisely because compiled libraries can participate in its ecosystem, but that integration leaves native compatibility obligations in the deployed application. A Python-only reading of the dependency graph cannot describe all of them.

Python 3.9.3 was itself expedited for security content, according to the release announcement. That context explains why reverting to an earlier installation is not equivalent to taking the hotfix. Python 3.9.4 restores the binary layout while providing the maintenance path recommended by the release team. For the reported 32-bit failure condition, the important result is that the interpreter again agrees with the thread-state declaration used by existing 3.9.x extension binaries.