October 20, 2018·4 min read·engineeringpythonconcurrencytimeouts

Python 3.7.1 uses elapsed time consistently for executor deadlines

Python 3.7.1 corrects timeout calculations in concurrent.futures and protects asyncio task enumeration. Other maintenance changes address process lifetimes and locks around fork.

A timeout describes how long an operation may wait. Computing it with a clock that changes when the system time is adjusted makes that duration depend on something outside the operation. Python 3.7.1 corrects this mismatch in concurrent.futures, using a monotonic clock for Executor.map() and as_completed() timeouts.

The October 20 release is the first final maintenance update to Python 3.7. Its included second-candidate changelog identifies bpo-34819 as the timeout correction. The notes state its purpose directly, preventing timeouts from deviating when the system clock changes. This is a change to the implementation of an existing contract, so application source can benefit without adopting a new API.

I value Python's concurrency libraries when their interfaces let an application express work without reproducing scheduling machinery. That convenience depends on details such as the clock used to enforce a deadline. A duration supplied by application code should retain its meaning when the machine's civil time changes.

A duration needs a clock with the right behaviour

The time module documentation in this release defines monotonic() as a clock that cannot go backwards and is unaffected by system-clock updates. Its reference point is undefined. Only differences between readings have meaning for measuring elapsed time.

This fits timeout calculations because the program needs a difference, not a calendar date. A deadline calculated from one monotonic reading can be compared with later readings from the same clock. Adjusting the system clock does not alter the duration represented by that difference.

from time import monotonic

started = monotonic()
deadline = started + 5.0
remaining = max(0.0, deadline - monotonic())

This illustrates the clock relationship, rather than replacing the executor's own implementation. The value of deadline is meaningful relative to later readings of the same clock. Treating it as an epoch timestamp for a log or external protocol would assign it a meaning the documented clock does not supply.

The changelog makes the affected library boundary specific. Executor.map() and as_completed() now use this clock for their timeout calculations. It does not claim a universal rewrite of every timeout in Python. The same maintenance series' first-candidate notes also record a monotonic-clock change in multiprocessing, showing another place where an existing wait had depended on the system clock.

These entries concern waiting behaviour, not a new guarantee that all work finishes before an exact instant. Scheduling and the operation's own semantics still matter. The correction removes clock adjustment as a source of deviation in the calculations the notes identify.

Task enumeration must tolerate task lifetimes

The final release changelog adds protection for weak-set manipulation in asyncio.all_tasks(). The tagged tasks implementation shows the mechanism. It first converts the global task collection to a list and then filters that list for unfinished tasks belonging to the selected loop.

That separates traversal of the task registry from the subsequent filtering. Task enumeration is itself runtime work, performed while task objects have their own lifetimes. The correction addresses how the enumeration obtains the collection it will inspect, rather than changing the application's coroutine definitions.

The result also has a defined scope in the implementation. all_tasks() excludes tasks that are done and selects the requested event loop. A caller using it for inspection should understand those criteria instead of interpreting the set as a permanent inventory of every task ever created.

Process operations need consistent shared state

The second-candidate changelog includes a logging change around fork(). Python acquires commonly used internal logging locks while forking to avoid deadlocks in the child. This is another case in which a library's internal state affects work outside the immediate API call. A child process inherits a state established while the parent may have multiple threads.

Multiprocessing also repairs a reference issue that could keep a pool alive after deletion when it had not been explicitly closed or terminated. That entry describes a lifetime bug. It does not make explicit shutdown unnecessary, but it corrects an unintended reference relationship in the implementation.

The same notes fix self-cancellation in the C implementation of asyncio.Task. This is distinct from enumerating tasks, since it concerns an individual task's transition toward cancellation. A maintenance update can therefore affect both how inspection observes tasks and how those tasks respond when their own execution requests cancellation.

My assessment of this maintenance release depends on these concrete behaviours. Deadline calculations use a clock suited to elapsed time, task inspection obtains a protected collection before filtering, and process operations receive corrections to shared-state handling. These changes make existing library operations closer to the assumptions their callers already express through durations, task queries and process lifetimes.