January 24, 2024·4 min read·engineeringcassandragossipupgradesmaintenance

Cassandra 4.0.12 corrects cluster version detection

Cassandra's January maintenance release fixes how gossip distinguishes an old-major upgrade from a patch upgrade. Related corrections address startup checks and stale endpoint state.

A cluster temporarily running two patch versions is not necessarily a cluster upgrading from Cassandra 3. Cassandra 4.0.12 corrects version-detection logic that could confuse those situations. The distinction matters because compatibility code uses the detected cluster state to decide which checks and behavior are appropriate during an upgrade.

The release record dates 4.0.12 to January 24. Its versioned changelog includes several related gossip and startup corrections. These are maintenance changes to the system's understanding of its peers, rather than new database capabilities exposed through CQL.

I am drawn to Cassandra for active-active operation at scale, while remaining cautious about introducing it where that requirement does not exist. A distributed database's maintenance behavior is part of the cost of that choice. The cluster has to interpret incomplete and changing membership information correctly while operators move between versions.

A patch upgrade looked like an older major version

CASSANDRA-18999 describes Gossiper.hasMajorVersion3Nodes() returning true during a patch upgrade even when the cluster contains no Cassandra 3 nodes. The issue includes a regression test with two live endpoints reporting adjacent patch versions from the same release family.

That test isolates the false inference. An upgrade may be in progress, but a difference between current and previous patch versions does not establish the presence of an older major release. Code asking the narrower question needs a narrower answer. Otherwise, compatibility behavior can remain active for reasons unrelated to the condition it is meant to detect.

The release corrects that detection. The practical importance is not that every patch upgrade previously failed. Its issue establishes a specific boolean result that was wrong under a specific cluster state. Identifying the bad result lets the maintenance discussion stay precise about which assumption changed.

Version detection also has to handle peers whose information is not yet complete. Treating every uncertainty as a permanent older-major condition can prevent the cluster from converging on the behavior appropriate to its actual membership. The related fixes show why endpoint lifecycle and version information need to be interpreted together.

Startup checks must match peer capabilities

CASSANDRA-18968 reports startup connectivity warnings when a Cassandra 4 node joins a cluster containing Cassandra 3 nodes. The issue attributes the failure to the startup check's PING messages not being available in the older version. A timeout in that case does not carry the same meaning as a supported peer failing to answer.

The 4.0.12 changelog says the connectivity check now runs only when no nodes are running a version earlier than Cassandra 4. This preserves the relationship between a check and the protocol capability it assumes. A diagnostic should not report inability to complete an exchange as evidence of a broken connection when the remote version cannot perform that exchange.

That does not eliminate the need for connectivity during an upgrade. It changes which diagnostic Cassandra can validly use in a mixed-major cluster. The narrower check avoids an expected compatibility condition appearing as a connectivity failure.

I find this distinction useful because operators make decisions from startup output. A misleading warning can send an investigation toward network policy while the actual limitation is a version-dependent message. Maintenance that improves the meaning of a warning reduces that ambiguity.

Departed endpoints should not keep version state unresolved

Another correction skips version checks for endpoints in a dead state. CASSANDRA-19187 explains how an administrative removal can leave an endpoint entry without ordinary version information. If the version supplier continues treating that entry as an unresolved current peer, it cannot retain the computed result as intended.

The issue describes repeated work acquiring a shared lock, with native request threads and gossip activity affected by the same synchronization. The administrative operation therefore has consequences beyond its immediate membership change. An endpoint that should no longer influence the version calculation can keep a common path doing synchronized work.

The shipped correction addresses the endpoint's lifecycle state before relying on its version information. Missing information about a departed node should not be interpreted like missing information about a live member that still needs to participate in compatibility decisions. That distinction is the mechanism behind the performance relevance of the fix.

Metadata snapshots need matching versions

The release also retrieves keyspace metadata and schema version consistently for DESCRIBE. CASSANDRA-18921 identifies two unsynchronized reads that could observe different schema states if a concurrent change occurred between them. The resulting mismatch could disrupt paging or make it inconsistent.

A version label only helps a consumer if it identifies the metadata supplied with it. Reading each independently can create a pair that never existed as one schema state. The correction makes the relationship between the snapshot and its version part of the retrieval operation.

These changes concern several different observations of distributed state. Patch differences should not imply an older major version, departed nodes should not keep live compatibility calculations unresolved, and a schema version should match its snapshot. For a Cassandra deployment that needs the system's distributed write model, 4.0.12 improves the maintenance behavior around those observations without requiring an application to adopt a new CQL feature.