Reaper 3.7.1 releases the replica locks it actually acquired
The memory backend gains segment coordination and fixes a mismatched lock-removal key. The regression test verifies that another segment can acquire replicas after release.
A scheduler can report that it released a lock while leaving the actual lock entry behind. That is the defect addressed in Reaper 3.7.1, released on 6 January. The patch fixes replica-lock release in the memory backend alongside changes that make this backend participate in repair-segment coordination.
Cassandra interests me because its distributed model addresses demanding availability requirements when the application actually needs them. Repair coordination is part of operating that model. My attention here is on Reaper's scheduling state, rather than a change to Cassandra's replication or storage engine.
The removal key did not identify the replica
PR #1535 shows the release bug directly. The old implementation began with each replica and constructed its lock key using the replica and repair-run identity. It fetched the associated lock information and checked the run identity.
The eventual removal then constructed a different key. It supplied the run identity's string representation where the replica identity belonged. The code had located one entry but attempted to remove another. Its validation of the fetched lock did not correct the identity used for deletion.
The replacement maps the supplied replicas to their lock keys and removes those keys directly. It also logs the keys being released and the remaining replica-lock keys. Acquisition and release now use the same kind of identifier at the map boundary.
This is a useful example of why apparently defensive checks do not establish correct cleanup. Checking ownership of a fetched object answers whether that object belongs to a run. It does not prove that a newly constructed deletion key points back to the same object.
A leftover lock can prevent another segment from acquiring the same replicas while it remains active. The patch's tests establish that behavior without relying on a timing measurement. The meaningful outcome is whether another acquisition succeeds after release.
The regression test checks reacquisition
The modified test first acquires locks for a repair run, segment and replica set. It then tries to acquire the same replicas for another segment in that run and expects failure. This establishes that the lock is effective before cleanup.
After release, the test repeats acquisition using a new segment identity and expects success. That final assertion is the part that detects the removal-key defect. Merely checking the boolean returned by release would accept an implementation that reported success without deleting the relevant entries.
The test therefore exercises a lifecycle. Acquisition establishes exclusion, a competing acquisition observes it, and release makes acquisition possible again. Each result constrains the meaning of the next operation instead of treating the release method's return value as sufficient evidence.
My preference is for that form of regression test in scheduler code. The next worker's ability to proceed is the user-visible consequence of cleanup. A lock-release implementation is only useful when the state it leaves permits the intended next operation.
Coordination moves into the common storage interface
The companion PR #1533 moves replica-lock operations from the distributed-storage interface to the common storage interface. The operations include acquiring, renewing and releasing locks, together with obtaining locked segments for a run.
That interface move changes the segment runner's behavior for the memory backend. In the affected repair branch, it previously bypassed the distributed-storage operation and returned success for a non-distributed backend. The new code invokes the common storage operation instead.
Memory storage delegates those methods to ReplicaLockManagerWithTtl. Its construction supplies a lead lifetime, with the facade defining a default of 90,000 milliseconds. This adds an expiring coordination mechanism inside that backend, rather than requiring every caller to invent separate memory-only behavior.
The interface move also affects orphan detection. Repair-manager code obtains locked segments through the common storage API instead of substituting an empty set for non-distributed storage. A backend can now supply actual segment leadership information where the caller previously had no such evidence.
Repair modes retain separate leadership paths
The segment-runner diff preserves a distinction for incremental repair without subrange incremental repair. That branch continues to use the distributed backend's lead methods when such a backend is present. Other affected branches use replica-lock methods on common storage.
This limits the claim that should be made about the release. It does not replace every leadership mechanism with one universal memory lock. It extends the relevant replica coordination operations to the common interface while keeping the existing mode distinction visible.
The release is also specific about backend scope. The incorrect removal key lives in the memory lock manager. Nothing in these changes establishes that every Reaper storage backend had the same cleanup bug or that Cassandra itself failed to release a database lock.
For operators using the affected backend, the result is concrete. Segment coordination no longer automatically succeeds by bypassing the storage operation, and releasing the replica keys permits another segment in the run to acquire them. The regression test verifies that transition after the cleanup call.