CockroachDB 2.1.3 distinguishes unavailable nodes from dead nodes
The December release repairs replica-count decisions during node outages and delays splits that would multiply pending snapshot work during bulk operations.
A node that is temporarily unavailable is not necessarily a node the allocator should exclude from its model of cluster capacity. CockroachDB 2.1.3 repairs a replica-count decision that lost that distinction. Under certain outages, the allocator could incorrectly reduce replication and potentially cause unavailability. The release also addresses a separate way that recovery work could multiply during imports and restores.
The December 17 release notes identify five-node clusters and high replication factors as affected cases for the replication defect. They also describe degradation with many pending Raft snapshots during bulk operations. The patches explain the mechanisms more usefully than a general claim of better resilience.
I value CockroachDB's use of consensus for consistency, while accepting the latency it requires. That preference makes allocator behavior important to the overall system. A consensus algorithm operates on the membership it is given. The machinery deciding how that membership changes during a failure needs a correct model of the nodes it is considering.
Temporary unavailability changes the count incorrectly
Alex Robinson's patch series in pull request 33047 first refactors the needed-replica calculation around available nodes. Its correction then includes nodes that are not live but have not been declared dead. Nodes being decommissioned remain excluded. The relevant count therefore depends on several states, not one Boolean live flag.
The patch description identifies the failure when the desired replica count exceeds the number of nodes currently running. The old logic could incorrectly down-replicate a range. Treating a temporary loss as a reason to lower the target can change the replication state during the very event the extra replicas were intended to tolerate.
This does not mean that dead nodes and temporary outages should be treated identically forever. It means the allocator's input must reflect the state distinction its decision requires. A node awaiting a definitive failure classification should not automatically disappear from every capacity calculation simply because it cannot currently respond.
The tests add an explicit unavailable-node category alongside live, dead and decommissioning categories. That fixture change matters because a test with only live and dead nodes cannot express the intermediate condition. The defect depends on the difference between current responsiveness and the node's classification for replica planning.
A replication factor is also not a count of healthy physical machines by itself. It is a target applied to individual ranges under a placement configuration. The allocator must reconcile that target with available nodes while preserving the intended behavior during transitions. The release repairs a particular case in that reconciliation, not the meaning of every possible topology.
Splits can multiply pending snapshot work
Tobias Schottdorf's patch in pull request 33015 addresses bulk-operation degradation. Its comments describe splitting a range whose replica already needs a snapshot. The split creates two ranges in that state. Repeating the process can multiply snapshot work before the cluster finishes the work already pending.
The patch delays eligible manual splits when they would produce more snapshot demand. It targets splits outside the ordinary split queue, including those associated with bulk operations. This is a sequencing control. It gives existing replica catch-up work an opportunity to progress before the operation creates additional work of the same kind.
That response is more precise than generally disabling splits or claiming that snapshots are undesirable. Splits and snapshots each have a necessary role. Their interaction becomes problematic when the rate of creating snapshot obligations outpaces the system's ability to satisfy them. Delaying the relevant operation changes that interaction without removing either mechanism.
The configured delay is bounded, and the helper checks Raft progress. A pause in a bulk operation can therefore be intentional behavior aimed at reducing a growing backlog. Treating every such pause as wasted time would miss the resource pressure the operation can create for the rest of the cluster.
My judgement is that this is an important part of a bulk operation's design. Import throughput is not the only useful measure if an import also creates recovery obligations that interfere with ordinary service. Controlling how quickly those obligations accumulate can improve the cluster's ability to make progress while the bulk operation continues.
Availability depends on transitions as well as steady state
The two repairs concern different decisions made during changing cluster conditions. Replica accounting needs to distinguish a temporarily unavailable node from a dead or decommissioning one. Bulk splitting needs to consider replica catch-up already in progress before dividing more ranges. Both decisions depend on state that a simple node count or data-size estimate leaves out.
For operators using the affected topologies or bulk workflows, 2.1.3 supplies specific repairs to those paths. The allocator retains the required unavailable-node distinction, and eligible splits can wait for snapshot progress instead of multiplying pending work immediately. These changes address how the database maintains its replication state while nodes and data organization are changing.