Cassandra 3.11.17 protects cleanup during range movement
The April maintenance release backports a safeguard against deleting newly streamed data during topology changes. It also makes concurrent client-warning collection safe.
Data arriving during a node decommission can already be on disk before its receiving node has final ownership of the corresponding token range. Cassandra's cleanup operation must account for that transition. Otherwise, it can interpret newly streamed data as unnecessary and remove it while the topology change is still in progress.
Cassandra 3.11.17, dated April 16 in its release record, backports a safeguard for that situation. The release changelog identifies CASSANDRA-18824 as the work bringing an existing correction to the older release line. A routine maintenance version therefore changes the safety of an administrative operation, even without introducing a new application query feature.
I am interested in Cassandra when active-active operation and scale justify its operational model. That model includes moving ownership while nodes continue participating in the cluster. Maintenance commands need to respect the intermediate states as well as the stable topology an operator eventually wants.
Pending ownership is different from unnecessary data
The backport issue describes cleanup calculating ownership from local ranges without considering pending ranges. During decommission, receiving nodes hold incoming ranges in a pending state while the transfer proceeds. Data that has reached their SSTables can therefore be valid transitional data even if it falls outside the calculation's current local ranges.
The issue gives a two-node reproduction with replication factor one. One node begins decommissioning while cleanup repeatedly runs on the receiving node. After the decommission finishes, some rows are missing because cleanup removed data the receiving node needed to retain.
That example is specific enough to explain the danger without assuming every cleanup loses data. The failure requires overlap between cleanup and a topology transition with pending ranges. The data's presence on disk does not, by itself, mean the ownership calculation already includes it.
The original report also emphasizes the misleading outcome. Streaming can finish without the topology operation reporting that cleanup removed its data. A successfully completed decommission is therefore not evidence that the overlapping cleanup was harmless.
The safeguard refuses an unsafe overlap
The backport discussion considers two responses. Cleanup could be redesigned to include pending ranges in its ownership calculations, or it could refuse to run while pending ranges exist on the node. The issue describes the latter as a smaller, more readily implemented safeguard, matching the correction already made in the 4.x line.
Refusing the operation preserves data until the ownership transition has settled. It does not attempt to make every combination of topology change and cleanup safe concurrently. That is a useful operational contract because it prevents a destructive interpretation of incomplete ownership state.
An operator can then sequence the work around the actual state transition. Cleanup is not merely a space-management action whose timing is irrelevant to the cluster. Its decision about which data is unnecessary depends on topology information, and pending ownership is part of that information.
I consider this a valuable backport because older deployments still need correct administrative behavior. The release does not require an application to adopt a new data model to benefit. It places a guard at the command that would otherwise act on an unsafe view of ownership.
Warnings also need concurrency protection
The same release changes client-warning storage to a thread-safe list. CASSANDRA-19427 traces rare ArrayIndexOutOfBoundsException failures to concurrent additions to an ordinary ArrayList. The reported query uses SELECT ... WHERE ... IN across multiple partition keys, with several reads local to the coordinator.
Those local reads can exceed the tombstone warning threshold and add warnings to the same ClientWarn.State. The query's work is concurrent even though the eventual warning collection belongs to one client operation. Treating that collection as single-threaded overlooks the execution path that contributes to it.
The reported correction uses CopyOnWriteArrayList. This addresses concurrent mutation of the shared collection. It does not change whether a query deserves a tombstone warning, and it does not make the underlying tombstone-heavy read cheaper. It keeps the diagnostic path from failing while reporting the condition.
The issue is also useful about diagnosis. The initial exception appeared inside a runnable, and the visible stack did not immediately identify the warning collection. Java Flight Recorder exposed the underlying ArrayList.add() exception, which connected the symptom to concurrent warning production.
Maintenance covers transitions and reporting
The cleanup fix protects data during a topology transition. The warning fix protects the reporting path when one query dispatches concurrent local reads. Both involve an operation that appears singular to its caller but depends on multiple pieces of internal state progressing at different times.
That relationship is relevant to the cost of operating Cassandra. Administrative completion and query response handling need invariants beyond a successful low-level read or stream. Cleanup must not discard data whose ownership is pending, and concurrent readers must be able to contribute warnings safely. Version 3.11.17 brings those corrections to deployments remaining on the 3.11 line.