Preserving parallel bulk execution through task_scheduler

A scheduler wrapper can preserve the ability to submit work while losing the ability to submit that work efficiently. Eric Niebler identifies that problem in P3927R0, published on January 15 and dated January 14. A task_scheduler wrapping a parallel_scheduler does not currently parallelize a bulk sender as the underlying scheduler would. The proposed change makes the wrapper preserve that customization.

The issue fits my mixed feelings about C++. Its abstractions can remove runtime costs, but understanding their interactions asks a lot of the reader. A wrapper that changes the execution strategy without changing the apparent algorithm is exactly the sort of interaction that makes a large codebase difficult to understand. This paper proposes a concrete repair to that mismatch.

Why the scheduler exists in two forms

The paper explains that std::execution::task normally stores its current scheduler in the type-erased task_scheduler. Type erasure lets a coroutine refer to a scheduler without making every concrete scheduler type part of that reference's static type. The resulting common interface needs to preserve the operations that matter to users of the original scheduler.

The related parallel scheduler proposal, P2079R10, describes the wider motivation. Independent components can otherwise create separate thread pools and oversubscribe the machine. A shared execution context gives those components a way to submit CPU work to common resources. Its replaceability design also allows integration with another parallel runtime.

That goal makes the wrapper problem consequential. Sharing an execution context is useful only if the route through a common interface retains the scheduling behavior that makes the context useful. An application should not need to recover a concrete scheduler type merely to regain the intended implementation of an algorithm.

A bulk operation carries information a single submission lacks

Submitting a range of independent work items gives a scheduler information it does not receive from one ordinary scheduling request. It knows that several items belong to a single operation. That information creates an opportunity to partition the range and coordinate completion as a group.

P2079R10 defines a backend interface with ordinary scheduling and separate chunked and unchunked bulk operations. A bulk receiver proxy accepts a beginning and ending index. That arrangement allows the frontend to describe work while the backend controls how it uses the execution context's resources.

The distinction is practical even without claiming a particular speedup. A scheduler that receives only an ordinary submission has less information than one that receives a bulk shape and a way to execute subranges. Losing the bulk operation at a wrapper boundary can therefore lose an implementation choice. No benchmark number is needed to establish that the two interfaces carry different information.

Reusing an existing backend

Niebler proposes replacing the wrapper's exposition-only shared_ptr<void> member with a pointer to parallel_scheduler_backend. A concrete scheduler adapter implements that backend. A completion domain then transforms bulk senders so they dispatch through its chunked or unchunked operation.

The adapter reconstructs a bulk sender using the concrete scheduler as its value completion scheduler. That restores the route to the scheduler's own domain customization. The design therefore preserves access to the original scheduler's implementation behind the erased interface.

This is the interesting architectural choice in the proposal. A new wrapper-specific parallel execution system would create another implementation to reconcile with the existing one. Reusing the backend keeps the relationship explicit. The wrapper exposes the operations needed to reach the concrete implementation, and the concrete implementation remains responsible for scheduling them.

Completion behavior belongs to the wrapper contract

The wording in P3927R0 forwards an upstream error or stopped completion unchanged. A successful value completion supplies the data needed for the bulk operation. That distinction matters because scheduling an algorithm is not the only way a composed asynchronous operation can finish.

A wrapper that preserves execution strategy also needs to preserve the meaning of completion. Otherwise callers might regain parallel work while acquiring different error or cancellation behavior. The proposal describes both the dispatch path and these completion paths, which makes its intent assessable beyond the headline performance problem.

RAII remains the part of C++ I value most, because scope gives resource ownership a concrete structure. Asynchronous interfaces need an equally clear account of operation lifetime and completion. The connection and start operations in this paper are part of that account. They explain when the backend begins handling the work and how it reports the result.

A proposal about a specific abstraction boundary

P3927R0 reports an implementation in NVIDIA's CCCL library. That is evidence of an implemented design, not a claim that every standard library already has the proposed behavior. As of this publication, the paper asks the concurrency and library working groups to change the specification.

For a reader evaluating the proposal, the useful question is whether the erased scheduler preserves the capabilities that its clients expect. An ordinary scheduling operation and a bulk operation impose different requirements on that common interface. The paper identifies the missing path and gives a mechanism for restoring it.

This should also temper expectations about parallel execution generally. Preserving a customization gives the underlying scheduler the opportunity to use its parallel implementation. It does not establish a speedup for an arbitrary loop or make shared data accesses safe. The proposed improvement concerns the information and dispatch behavior retained when a coroutine's current scheduler passes through task_scheduler.