Python 2 support ends with the compatibility problem still visible

Python 2 reaches its support deadline today, January 1, 2020. The distinction between a support deadline and a release date has already caused confusion. In the 2018 python-dev discussion of the deadline, Benjamin Peterson explained that the timing of a final release would depend on the availability of the people producing it. The conservative assumption was that changes landing after December 31, 2019 would never be released. An interpreter can still execute an application after that boundary. What changes is the expectation that upstream developers will continue maintaining it.

That matters because compatibility has been the central constraint on this transition. An application can remain valuable while depending on assumptions that its replacement runtime deliberately rejects. Converting its syntax does not establish that the application still interprets its inputs correctly. The support deadline forces attention onto those assumptions without making the conversion any more mechanical.

I value Python principally for its libraries and the work they make available. Its packaging and dependency management are a substantial cost of that choice. The end of Python 2 makes that tension explicit, because the same dependency graph that makes a program useful can prevent it from moving to a supported interpreter.

Why maintenance lasted this long

The April 2014 update to PEP 373 extended Python 2.7 maintenance to 2020. Its stated purpose was to clarify the position for users who could not yet migrate and for vendors supporting Python 2. It did not promise a regular schedule of bugfix releases. It also ruled out Python 2.8. Continued maintenance therefore meant preserving an existing branch while Python 3 developed, with specific exceptions where maintaining the old behaviour had become a problem.

One of those exceptions was network security. PEP 466 allowed selected security capabilities to move from Python 3.4 into Python 2.7 maintenance releases. These included constant-time comparison in hmac, password hashing support in hashlib, and substantial changes to ssl. The proposal treated secure networking as a reason to relax the usual restriction on adding features to a maintenance branch.

Its rationale explains why simply telling everyone to migrate was inadequate. Applications could depend on Python 2 through platform infrastructure, and fixing their Unicode handling was substantial work. Requiring that entire migration before improving their network security would leave the security problem unresolved. The proposal also rejected creating Python 2.8 because another feature release would introduce disruption without delivering the low-impact update that existing installations needed.

This was a considered accommodation of deployed software. It also demonstrates the recurring cost of preserving two branches. Security changes needed an explicit scope, compatibility rules and their own backport work. Indefinite compatibility would require people to keep doing that work whenever assumptions in the old runtime became inadequate. The maintenance extension bought migration time, but it did not remove the differences between the languages.

Text and bytes explain much of the difficulty

Brett Cannon's porting guide as shipped with Python 3.8.0 identifies text and binary data as a major area that cannot be completely automated. In Python 2, str can represent either. Python 3 separates Unicode text in str from binary data in bytes. Code that passes a string through several layers must now make its intended meaning explicit.

For a network response containing UTF-8 text, a small Python 3 function can express the boundary directly. The following example uses the decode and encode operations described in the porting guide. The assertions specify the intended interpretation of the input and the bytes produced at the output.

def normalise_label(payload):
    text = payload.decode("utf-8")
    return text.strip().encode("utf-8")

assert normalise_label(b"  worker  ") == b"worker"
assert normalise_label(b"  caf\xc3\xa9  ") == b"caf\xc3\xa9"

The application performs its text operation between decoding and encoding. This follows the guide's recommendation to decode incoming binary text near the boundary and encode outgoing text as late as possible. The explicit encoding is part of the function's contract. A source translator cannot infer it merely from the fact that a Python 2 function accepted str.

The same distinction affects APIs that appear unrelated to encoding. The guide gives the example of indexing binary data. In Python 2, b"123"[1] produces a one-character string. In Python 3, it produces the integer 50. A byte parser that passes that result onward can therefore change behaviour without changing the indexing expression. Slicing has different compatibility implications, so replacing expressions indiscriminately obscures the intended operation.

Constructors also require care. The guide contrasts Python 2's bytes(3), which produces the textual representation of the integer, with Python 3's allocation of three zero bytes. These are valid expressions with different results. A successful import gives no evidence that the application still produces the same protocol message or file content.

I prefer the explicit distinction between text and bytes. It gives an API a more precise contract. The migration cost is that an old program may never have recorded that contract, leaving its maintainers to recover meaning from callers and data formats.

Compatibility includes arithmetic and protocols

Division is another case where source code can remain valid while its result changes. The porting guide explains that Python 3 evaluates 5 / 2 as 2.5. Python 2 code expecting integer division needs a deliberate decision about whether the operation means true division or floor division. Blindly replacing every slash with // would make a different set of assumptions.

The guide gives a protocol-level reason for avoiding that replacement. An object may implement __truediv__ without implementing __floordiv__. The operators therefore select different supported operations, even when a particular example happens to involve integers. This is especially relevant to Python's object model, where operators can dispatch through methods supplied by a library.

A compatibility change has to preserve the intended operation as well as a representative result. For integer quantities, a calculation may require a whole number of complete groups. A different calculation may require a fractional ratio. Tests that happen to divide evenly cannot distinguish those contracts. The guide recommends introducing future division in Python 2 and then deciding where floor division is actually required.

Python's protocol model is one of the parts of the language I respect. Libraries can participate in ordinary language operations without requiring every caller to use a separate API. It also means a migration cannot safely reason about every operator as if it only acted on built-in numbers.

The dependency graph determines when a port can run

The porting guide separates modernising source from being able to run the complete application on Python 3. It recommends starting the source work even when dependencies still block execution. Its discussion of caniusepython3 covers dependencies that block a transition directly or indirectly. That distinction matters because importing one package can depend on compatibility work several packages away from the application itself.

The tools described in the guide make different compromises. Futurize aims to make Python 3 practices available under Python 2. Modernize targets a more conservative common subset and relies on six. Either approach can reduce mechanical editing, but the guide explicitly reserves semantic decisions for the developer. Supporting both runtimes is an engineering choice with continuing costs in testing and compatibility code.

Extension modules add a separate concern. The Python 3.8.0 extension-porting guide directs extension authors to resources specifically about C extensions. It also describes Cython and CFFI as abstractions that can handle differences between Python versions and implementations after an extension is rewritten to use them. An application whose Python files are compatible can still depend on native code that needs its own migration.

For a library maintainer, compatibility also needs to be communicated. The porting guide recommends declaring supported versions through package classifiers in setup.py and updating those declarations when Python 3 support works. That metadata describes a claim about the package. Continuous integration against the supported interpreters supplies evidence for it. Neither replaces the dependency work needed to make the complete installation usable.

The support boundary changes ownership

The deadline does not establish that every remaining Python 2 program is defective. It establishes a different maintenance position for programs that remain there. The 2018 release-manager discussion makes clear why the existence or timing of a final artifact cannot be treated as a commitment to accept further changes. Release production and continuing development are different activities.

PEP 466 had already distinguished upstream policy from what commercial redistributors might choose to support. A downstream arrangement must therefore be evaluated on its own terms. The upstream deadline cannot say which particular vendor will maintain an installation, and a vendor's separate commitment does not extend the upstream project's maintenance policy.

For applications moving to Python 3, the remaining work is concrete. Their interfaces need defined text and binary contracts, arithmetic must preserve its intended operation, and dependencies must support the selected runtime. Those are properties of the application and its libraries. The calendar makes the work more urgent while leaving its technical content intact.