Python's final 3.11 beta makes task lifetime explicit

Starting several asynchronous operations leaves an important question unanswered. If one fails, who waits for the others and decides whether they should continue? The TaskGroup API available in Python 3.11's final planned beta puts that decision into a context manager. Its contract covers the work after failure as well as the normal path through concurrent tasks.

The July 26 release announcement asks third-party maintainers to test Python 3.11 before the release-candidate phase. It describes a beta intended to prepare projects for the feature release and discourages production use. Task groups are already part of that testable implementation. Evaluating them requires looking at their failure semantics, not merely replacing a task-creation call.

I appreciate Python's context-manager protocol because a library can attach resource lifetime to an ordinary language construct. Task groups apply that useful mechanism to concurrent work. The resulting boundary makes ownership easier to read, provided the caller understands what happens when a child task fails.

A context owns the wait

The asyncio documentation at the beta tag defines TaskGroup as an asynchronous context manager. Tasks enter the group through its create_task method. Exiting the context waits for all tasks belonging to the group.

import asyncio

async def value(number):
    await asyncio.sleep(0)
    return number

async def main():
    async with asyncio.TaskGroup() as group:
        first = group.create_task(value(10))
        second = group.create_task(value(20))
    return first.result() + second.result()

assert asyncio.run(main()) == 30

The example keeps the task objects so that their results can be read after the context exits. The significant guarantee comes from that exit. On the successful path, both tasks have finished before the program reaches the return statement. The enclosing function does not have to maintain a separate loop whose job is to await every task it created.

The group is not limited to tasks created directly in the context body. Its documentation allows a coroutine to receive the group and add more tasks while the context is waiting. Once the last task finishes and the context exits, additions stop. The ownership boundary remains the group even when the task-creation sites are distributed across helper functions. Passing the group makes that relationship explicit in the helper interface. Returning from a helper does not transfer its unfinished tasks to an unspecified owner.

Failure changes the group's state

The first non-cancellation exception in a child task cancels the group's remaining tasks. The group then refuses further additions. If the context body is still active, it also cancels the task containing that body so that an awaiting operation can be interrupted.

That internal cancellation does not escape the containing context as a bare CancelledError in this case. The group first waits for its tasks to finish. It then combines non-cancellation failures into an ExceptionGroup or BaseExceptionGroup, as appropriate, and raises the result. The caller receives the failures after the group has completed its waiting policy.

Multiple failures remain relevant because cancellation and cleanup can overlap. Reporting only the first observed exception would discard information about other tasks that also failed. Grouped exceptions let the runtime preserve those failures in one raised object. The concurrency abstraction therefore has an error-reporting contract that matches the possibility of several unsuccessful children.

The body of the context participates in that policy too. If the body raises an exception, the group cancels and waits for the remaining tasks. Its non-cancellation exception joins the group of reported failures. An error in the parent's orchestration code does not leave the child tasks outside the cleanup boundary.

Two base exceptions receive special treatment. For KeyboardInterrupt and SystemExit, the group still cancels and waits for the other tasks, then re-raises the original exception. The documented policy preserves those control-flow meanings while retaining the same obligation to finish managing the children.

Why gather has different behavior

The same beta documentation describes a different default for asyncio.gather. It propagates the first raised exception immediately to the task awaiting it, while other submitted awaitables continue running. Setting return_exceptions=True instead collects exceptions alongside successful results.

Those are useful behaviors, but they express different policies. An application that wants independent operations to continue can use an operation whose contract permits that. A task group declares that a non-cancellation child failure should stop the remaining group work and that the context must wait for completion before it reports the grouped errors.

I prefer a task group when the operations belong to one bounded activity. That preference follows from its ownership rules, not from the smaller amount of code in the example. The correct choice still depends on whether a child failure should cancel its siblings.

Cancellation itself remains cooperative. The beta documentation says cancelling a task arranges for CancelledError to enter its coroutine, which has an opportunity to clean up and can even suppress the request. Waiting for a group therefore means waiting for the children to finish responding. The context defines responsibility for that wait without promising that a cancellation request instantly terminates execution.