Python 3.6.5 corrects cancellation and locking in asyncio

Cancellation becomes difficult when a coroutine is waiting for permission to proceed. Notification and cancellation can arrive close together, and the synchronization object still has to preserve its contract. A cancelled waiter must not quietly continue as though nothing happened. A lock must not grant ownership again while it remains held. Python 3.6.5 repairs failures in both of these areas.

The release dated March 28 is a maintenance update to Python 3.6. The useful detail is in the tagged release candidate changelog, whose changes are included in the final. Several entries concern the behaviour of running systems, including asynchronous coordination, subprocess startup and SSL transport shutdown. None requires an application to adopt a new language construct to benefit.

I value Python's libraries because they often provide the mechanism a program needs without much application code. That convenience makes the correctness of their coordination rules especially important. A short call to a synchronization object can depend on interpreter-library behaviour that the application cannot see at the call site.

A notification must not erase cancellation

The changelog records bpo-32841 as an asyncio.Condition correction. The earlier implementation could silently ignore cancellation after notification and cancellation of a conditional lock. This concerns the interaction of events that application code usually treats as independent, a change in the condition and a request to stop waiting.

Ignoring cancellation changes the caller's control flow. Code awaiting a result may continue into work its owner has already decided to abandon. The exact application consequence depends on the surrounding coroutine, so the release note does not justify claiming a universal failure mode. It does identify a violation at a useful level, cancellation could disappear during synchronization.

The same notes record bpo-32734, an asyncio.Lock safety problem that permitted acquisition and locking of the same lock multiple times without it becoming free. A lock's value depends on who may proceed while it is held. If that exclusion fails, correct-looking application code can enter a section under assumptions the library has not actually enforced.

These corrections affect different synchronization objects. Together they explain why maintenance of an asynchronous runtime cannot be assessed solely by checking that coroutines start and finish in a simple example. Ordering among waiters and cancellation requests is part of the library's behaviour. An application relies on that ordering even if its own source contains no explicit scheduling logic.

The changelog credits Bar Harel for both corrections. Naming the fixes also makes it possible to distinguish this release's work from a general recommendation to rewrite asynchronous code. The reported defects are in the existing standard-library implementation, and the patch repairs that implementation.

Closing a transport must stop its activity

Another entry, bpo-33037, says SSL transports skip sending and receiving data after closing. Shutdown is a state transition that affects what subsequent callbacks may do. Work already scheduled around a connection can otherwise encounter a transport whose lifetime has ended.

This change belongs in the same operational discussion as synchronization because a coroutine's lifetime and its connection's lifetime need to agree. Cancellation can end an operation, while transport closure ends the means of performing its network I/O. The changelog does not establish that these entries share one cause. They address separate implementation details around the end of ongoing work.

Python 3.6.5 also fixes a rare possible deadlock in the POSIX subprocess child before execution. The notes identify marking file descriptors inheritable on execution as the relevant activity. A parent that expects to launch another program depends on that preparation finishing before the child runs the requested executable.

The subprocess entry says the defect appears to have been introduced in 3.4. That history matters to maintenance decisions because the age of an API does not imply that every path through its implementation has settled. A patch release can correct a long-standing problem in code that applications have called for years.

Security fixes concern processing cost and memory

The release candidate notes also identify catastrophic backtracking in regular expressions used by difflib and poplib. Refactoring those expressions resolves CVE-2018-1060 and CVE-2018-1061. The reported problem is denial of service through matching behaviour, which makes the amount of work performed on an input part of the security boundary.

A separate minimal correction prevents a buffer overrun in os.symlink on Windows. The distinction between these entries matters. One addresses how a pattern matcher spends computation; the other addresses native memory handling. Describing the release merely as an asyncio update would omit security work that applies through unrelated standard-library calls.

The final changelog adds an update to bundled pip 9.0.3 and setuptools 39.0.1. Those versions describe the tools included in this interpreter release. They do not establish what an independently maintained environment already contains.

My assessment of 3.6.5 rests on the repaired contracts in the standard library. Applications can retain their existing syntax while receiving corrections to lock exclusion, cancellation propagation and transport closure. Those changes directly affect whether the program's existing control flow means what its author expects when tasks stop or compete for access.