PEP 744 sets conditions for CPython's experimental JIT

CPython's experimental JIT already exists in the development branch. The question raised by PEP 744 is what would justify making it a permanent part of the interpreter. Brandt Bucher's public discussion on April 12 asks for agreement on those conditions. This is an informational proposal about the implementation and its obligations, with deployment cost and maintenance considered alongside execution speed.

The original PEP revision makes the choice of compiler technique understandable. CPython needs a way to turn optimized operations into machine code without maintaining a large independent compiler backend. Copy-and-patch moves much of that backend work into the interpreter build. At runtime, the JIT assembles machine-code templates and patches the parts that depend on the particular trace being compiled.

I find this more interesting than a general claim that Python has acquired a JIT. Python's existing libraries are usually my reason for accepting the language's costs. Improving general interpreter execution without demanding a new programming model could benefit that choice. The proposal still has to explain how its implementation remains compatible with the interpreter and maintainable by the people developing it.

Specialization came before machine-code generation

PEP 659 describes the adaptive specialization that supplies part of the background. An interpreter can replace a general instruction with a form suited to the types and values encountered during execution. It can return to a more general form when those assumptions stop holding. The unit of specialization is small, which keeps the cost of abandoning an incorrect assumption manageable.

That means specialization and machine-code generation are separate decisions. Knowing that an operation usually receives a particular kind of value can help an interpreter execute it more efficiently. It also provides information that a later compiler can use. A JIT does not have to discover all the useful runtime information independently if the interpreter already records it.

PEP 744 describes the next stage as translating sequences into smaller micro-operations and optimizing traces of those operations. Interpreting the resulting trace still incurs overhead for dispatch and decoding. Increasing the number of small operations can make that overhead important even when the trace contains useful optimizations. Compiling the trace creates an opportunity to remove repeated interpreter work between its operations.

This is why compiling an optimized trace has a different purpose from translating each original bytecode instruction directly into machine code. The trace representation allows optimizations to consider work at a finer level. Its value depends on how effectively that representation can execute after optimization. A second interpreter for smaller operations does not automatically provide that execution advantage.

LLVM runs during the build

The distinction between build-time and runtime dependencies drew questions in the discussion. Bucher explains that LLVM compiles micro-operation implementations into machine-code fragments during the build. CPython then incorporates those fragments into the interpreter. At runtime, code generation copies the relevant fragments and adjusts the locations that require trace-specific values. It does not start LLVM to compile each trace.

That division gives the project access to an existing compiler's code generation while keeping the runtime mechanism smaller. It also places a particular obligation on the build. The fragments have to match the target architecture and configuration of the interpreter that contains them. Generated machine code is therefore different from a portable generated text file that could be reused indiscriminately across builds.

The original proposal explains that the interpreter and JIT draw from the same instruction definitions. Changing an instruction can consequently update generated components together. That reduces the amount of duplicated instruction behavior a maintainer must edit manually. Larger changes to interpreter state or error handling can still require work in the JIT template and supporting code. Shared generation reduces a source of divergence without proving that every generated execution path is correct.

Experimental status has operational meaning

The PEP proposes requiring a meaningful speed improvement on a popular platform before removing experimental status. It also requires a build and distribution process with limited disruption, followed by a Steering Council judgment that the benefit exceeds the costs. The initial performance threshold is on the order of five percent. These conditions are proposals for discussion, not a declaration that the JIT has already met them.

The document describes current performance as roughly comparable to the specializing interpreter and identifies executable-code memory as an additional cost. That makes the experiment's purpose more specific. It establishes an implementation on which further optimization can proceed. It does not establish that enabling the current implementation improves every Python program. The original PEP explicitly says experimental builds should not be used in production.

Memory matters independently of an execution-time result. Generated code occupies storage in addition to the application's data. A change that reduces interpreter work can still increase the resources needed by a process. The relevant tradeoff is therefore specific to the workload and the runtime configuration. Reporting an execution improvement without accounting for the extra code would leave part of that tradeoff unexplained.

Compatibility includes debugging tools

The original PEP expects Python behavior and third-party package interfaces to remain unchanged. It also says Python tracing and profiling facilities continue to work. Native debugging has a separate limitation, since C debuggers cannot yet walk through JIT frames with the same completeness as ordinary compiled frames. The document distinguishes examining a leaf frame from reconstructing the full native call stack.

That distinction matters for an interpreter with a substantial native extension ecosystem. Preserving the Python-level programming model is necessary, but diagnosing failures can also require understanding execution across native frames. A generated path that behaves correctly during ordinary execution can still make a difficult failure harder to investigate. Debugging support belongs in the discussion of what makes the implementation ready for wider use.

I regard shared instruction definitions as the strongest part of this design argument. They give the JIT a way to evolve alongside the interpreter instead of depending on a separately maintained account of its semantics. The remaining work is explicit in the proposal, including native debugging and the cost of executable memory. Consensus on the acceptance conditions will determine which of those obligations must be resolved before the experiment becomes a supported interpreter option.