Cassandra 4.0 RC1 makes upgrade compatibility explicit

Cassandra 4.0's first release candidate makes compatibility work a substantial part of evaluating the release. The changes reach beyond enabling new capabilities. Existing clients may use a beta protocol whose final wire format differs, repair changes the treatment of stored data, and configuration defaults can affect whether an upgraded node starts. A successful process restart is therefore a narrow observation about a much larger transition.

The RC1 distribution files provide the candidate's source and binary packages. Its tagged NEWS file documents the changes discussed here. RC1 is a release candidate, so these notes describe the implementation available for evaluation. They do not establish a generally available release or justify assuming that every deployment can adopt it unchanged.

I find Cassandra compelling when an application actually requires its approach to distributed operation. That qualification matters during a major upgrade. The operational commitments become more concrete when the release documents how old files, clients and configuration values interact with new code. RC1 supplies enough detail to identify several distinct compatibility boundaries.

A protocol version can outlive its beta encoding

Native protocol v5 moves out of beta in this release. The notes warn that its wire format has changed substantially and that clients using v5-beta need drivers supporting the final v5 format. The version label alone is insufficient to establish compatibility. A client that supported the beta format still has a specific migration requirement.

This warning concerns users connecting over v5-beta. It does not say that every client must immediately adopt v5. That narrower scope matters when discussing an upgrade across multiple applications. The relevant question is which protocol implementation a client actually uses, including whether it relies on the beta encoding. Treating all existing drivers as equivalent would conceal the condition that the notes explicitly identify.

Server-side DESCRIBE statements are another documented addition. Meanwhile, cqlsh gains Python 3 support and its startup script prefers python3 when selecting a runtime. Python 2.7 support is deprecated. These changes affect the administrative client independently of the protocol used by another application, so a working application connection does not prove every existing administration script behaves as before.

The shell also returns a nonzero exit code on errors. The notes call that backward incompatible because scripts may depend on the previous behavior. A script that interpreted process success without checking its query result can now take a different path. This is a useful change in error reporting, but it remains a change in the contract between Cassandra's tooling and automation.

Repair changes the meaning of repaired data

The release notes describe a correction to incremental repair that previously allowed repaired data to be inconsistent between nodes. The correction changes both full repair and incremental repair. Full repair no longer marks data as repaired. Incremental repair moves anticompaction to the beginning of the repair instead of the end.

These details matter because retaining an old operational description of repair can produce a mistaken interpretation of the upgraded system. A full repair and an incremental repair no longer have the same effects they had before this change. The notes specifically instruct users who previously used incremental repair to run a full repair after upgrading to resolve possible inconsistencies.

The release also adds ways to inspect repair work. A preview reports the streaming expected for a repair, while validation checks consistency of repaired data between nodes. The documented commands expose these as separate operations:

nodetool repair --preview
nodetool repair --validate

A preview addresses the work a repair is expected to require. Validation addresses the consistency of data already classified as repaired. Those observations answer different questions, and neither command should be described as a substitute for the corrective full repair specified in the upgrade notes. The distinction gives operators a more precise vocabulary for interpreting repair state.

Another new setting, repair_session_space_in_mb, lets operators reduce Merkle tree size when repair causes excessive heap pressure. It replaces the earlier tree-depth setting as the preferred configuration. The documented control is a memory allowance expressed in megabytes, which makes the intended resource constraint more direct than a depth parameter alone.

Defaults can alter startup and placement

RC1 changes the default num_tokens value from 256 to 16. The notes also say that allocate_tokens_for_local_replication_factor is uncommented and set to 3. These are defaults for the supplied configuration, not instructions to reinterpret an existing node's token ownership as if it were a new deployment.

The upgrade warning is explicit. An upgraded node refuses to start if its configured num_tokens differs from the value it requires. A new node joining with a different token count from the rest of its datacenter can also own a different amount of data. Copying the new default into an existing deployment therefore deserves attention as a placement decision, even when the numeric value looks like an ordinary tuning change.

Token configuration validation also requires num_tokens when a virtual-node configuration specifies initial_token. The count must match the supplied tokens. This puts a configuration error into the startup path, where automation will observe a node that does not start. The release's explanation helps distinguish a deliberate validation failure from an unexplained process failure.

Local system-keyspace placement changes as well. With specified table exceptions, keyspaces using a local strategy now store their data in the first data directory by default. The stated purpose is to tolerate failures of other disks. A separate system_data_file_directory setting permits storing this data in a directory that provides redundancy, and startup migrates the relevant local system data when necessary.

That behavior makes disk layout part of the compatibility analysis. Changing where local system data resides affects which directory needs protection. The notes explicitly exclude some tables, including system.paxos and system.compaction_history, so the change cannot be summarized as moving every system table into one place.

Old data formats require an intermediate version

RC1 removes support for pre-3.0 SSTable formats. A 2.x deployment cannot upgrade directly to 4.0. The documented route passes through a 3.x version and upgrades the SSTables there. This requirement concerns the files Cassandra must read, so merely changing the executable version along an acceptable path is insufficient if old-format files remain.

Other upgrade steps address schema and metadata. OldNetworkTopologyStrategy is removed, so affected keyspaces must use NetworkTopologyStrategy before the upgrade. Legacy authentication tables and their migration code are also removed. The notes require the earlier authentication migration to be complete and the old tables to be removed.

Replication options receive stricter validation. Creating or altering a keyspace with NetworkTopologyStrategy requires the named datacenters to exist in the cluster. Operators must add nodes in a new datacenter before setting a policy that refers to it. Existing keyspaces continue operating, but subsequent schema operations encounter the new validation. That difference matters to automation that establishes replication configuration before introducing the corresponding nodes.

Operational controls have changed semantics

The new full query log records queries, approximate invocation times, bind parameters and query options. It uses Chronicle Queue and provides controls for heap and disk utilization. The release includes fqltool to dump or tail a readable form. Those details make it possible to discuss the evidence the log captures without assuming that a query string alone represents the entire request.

Audit logging is a separate addition covering incoming CQL commands and authentication attempts, including failures. The notes also allow operators to restrict a login role's activity to selected datacenters. Changing a role from login-enabled to login-disabled prevents existing connections from executing statements after the relevant cache refresh. The timing qualification matters when interpreting the effect of an administrative change on connections that already exist.

Snapshot administration changes too. The notes require an explicit --all flag for nodetool clearsnapshot to remove every snapshot. The earlier command deleted all snapshots by default, so existing cleanup scripts need to account for this more explicit selection.

Some features remain explicitly experimental. The notes retain that designation for Java 11 and transient replication. Their presence in a release candidate does not remove the qualification. Similarly, creation of new materialized views and SASI indexes defaults to disabled, while existing ones continue working. These are separate decisions about admitting new use and preserving existing data structures.

I regard this candidate's documented compatibility work as central to its usefulness. A deployment can map its actual dependencies onto named changes, including the protocol encoding used by clients and the stored formats on disk. For clusters that have used incremental repair, the release also specifies the full repair required after the upgrade to address possible pre-existing inconsistencies.