June 6, 2022·4 min read·engineeringpythonsubprocesslinux

Python 3.10.5 repairs process creation in restricted environments

Python 3.10.5 falls back to fork when vfork fails. Its maintenance fixes show why optimized system calls need fallback behavior that respects the surrounding operating system.

A process-creation optimization can fail even when the application is allowed to start a child process. Python 3.10.5 repairs one such case. When subprocess tries vfork and the call returns an error, it now falls back to fork. The June 6 release includes this correction within an ordinary maintenance update.

The version-specific changelog explains the motivation. A kernel may disallow vfork, so failure of that implementation choice should not automatically prevent the operation. A separate entry repairs os.closerange potentially becoming a no-op inside a Linux seccomp sandbox. Both fixes concern behavior under restrictions that a less constrained development environment might never exercise.

I have little affection for Python's operational complexity, but these are useful maintenance changes. A library interface should retain its documented meaning when one of its preferred implementation mechanisms is unavailable. Otherwise an application acquires a dependency on a particular system-call policy without declaring one.

An optimization does not define the operation

The subprocess repair distinguishes the user's request from the mechanism selected to carry it out. A caller requests a child process. The implementation may attempt an optimized path, but that path's rejection does not establish that every permissible implementation must fail.

Falling back to fork preserves another route to the requested operation. This is narrower than bypassing the kernel's policy. The kernel still decides whether the alternative call is allowed. Python simply avoids treating an error from one attempted mechanism as the final answer before trying its supported alternative.

That distinction matters when reasoning about containers and sandboxes. The changelog specifically refers to a kernel disallowing vfork. It does not imply that a fallback can evade all process restrictions or that every sandbox allows fork. The useful guarantee is a repair to Python's behavior after the optimized call fails.

The release also adds a private subprocess._USE_VFORK attribute as a fail-safe. Its release note says there is no known need to disable the optimization, directs any caller doing so to set it only to false, and asks for a CPython issue explaining the reason. This is an escape mechanism for an unexpected compatibility problem, with a route for reporting the underlying defect.

Private configuration has a different maintenance status from the public process API. Treating that attribute as a general tuning option would ignore the purpose documented by its authors. The fallback is the relevant ordinary behavior in this release. The private control exists for a problem that the normal implementation does not yet cover.

Closing descriptors must still close descriptors

The os.closerange repair concerns a more direct contract. Its 3.10.5 documentation says it closes file descriptors beginning at the lower bound and ending before the upper bound, ignoring errors. The documented equivalent is a loop that attempts to close each descriptor in that interval.

A no-op under seccomp restriction fails that contract even though ignoring individual close errors is intentional. The distinction is between tolerating unavailable descriptors and failing to perform the operation over the range at all. Code relying on the function has already expressed which resources it wants closed.

This matters to resource ownership. A caller may use a range operation precisely because it cannot conveniently enumerate every open descriptor of interest. If the operation silently does nothing, successful return no longer supplies the expected cleanup behavior. The changelog's repair restores the operation in the affected restricted environment without requiring application code to reproduce the implementation.

The two changes also show why optimized system interfaces need separate failure reasoning. Choosing a faster implementation is useful only if the surrounding code handles the ways that implementation can be unavailable. Returning an error and returning without doing useful work are different failures, but both can invalidate the higher-level operation.

The interpreter's own process machinery

The subprocess implementation at the release tag gives context for this boundary. Before its native process-creation call, it prepares the executable candidates and environment data. Its comments explain that complex work involving allocation in the child can cause deadlocks, so it performs that preparation beforehand.

The code also uses an error pipe to learn whether execution in the child failed. It closes pipe ends in cleanup paths and bounds the amount of error data it reads. These details explain why process creation is more involved than selecting a system call. The wrapper must arrange resources around that call and communicate failures without leaving its own descriptors behind.

I prefer these concrete compatibility repairs to application-specific workarounds for interpreter behavior. The public operations remain familiar, while the runtime handles a documented restriction on its implementation. Python 3.10.5 supplies both the process-creation fallback and the descriptor-range correction at the layer that owns those decisions.