Cassandra 5.0 RC1 tests indexing against replica disagreement

Two replicas can each hold part of a row that matches a query after reconciliation, while neither replica sees a complete local match. An index that discards both partial matches can prevent the coordinator from ever reconstructing the right result. Cassandra 5.0's first release candidate includes work to make storage-attached indexing preserve the information distributed reads need.

The release record dates 5.0-rc1 to July 10. Its tagged changelog contains many stabilization changes, including CASSANDRA-19018's correction for SAI intersections over unrepaired data at consistency levels requiring reconciliation. The candidate makes the developing release available for evaluation. It is not a general-availability announcement.

I am interested in Cassandra for the distributed write model it supports when a workload needs it. Richer indexing is valuable in that setting only if the index's local decisions remain compatible with distributed reconciliation. This part of the candidate explains more about that requirement than a list of new query operators would.

A local intersection can hide a global match

CASSANDRA-19018 gives a small example with primary key k and ordinary columns a and b. One replica holds a=1 for a row, while another holds b=2 for the same key. A query asks for rows where both conditions hold, using a consistency level that reads and reconciles multiple replicas.

At the coordinator, the partial information can combine into a row with both requested values. At either replica, however, a strict intersection of local index matches finds no complete match. If both replicas return nothing, the coordinator lacks the evidence needed to produce the result.

The issue's test uses separate internal writes to construct that divergent state deliberately. Those test operations are a way to expose a distributed condition, rather than application guidance for normal writes. Repairing the test data first makes the original reproduction succeed because the replicas then have the complete information locally.

That last detail identifies the relevant dependency. An index can be correct against a repaired local representation and still make the wrong filtering decision against a partial representation. The query's distributed consistency requirement needs to influence how aggressively replicas discard candidates.

Preserve candidates before final filtering

The implementation discussion separates repaired and unrepaired data. For unrepaired data, it can retain partial index matches by using a union where a strict intersection would discard them. The coordinator's replica filtering protection then participates in resolving and filtering the resulting rows.

This is not equivalent to replacing every AND in an application query with OR. The user's predicate still asks for the intersection. The change concerns an intermediate candidate set that must include enough information for the final predicate to be evaluated against reconciled data.

That distinction is central to the cost. Retaining more candidates can require extra reads and data transfer. A replica cannot always decide a row is irrelevant using only its local partial view, so the system sometimes has to do additional work before it can safely reject it.

The issue's development history also shows why an initial solution needed more than changing an iterator. Local post-filtering could discard the same partial match later. Coordinator protection had to handle paging and deletes, and static-column matches could require fetching more of a partition. The correction spans the places where a candidate might otherwise disappear before reconciliation.

I find that history useful because it makes the performance tradeoff concrete. Correct distributed filtering can require carrying extra information through several stages. The amount of unrepaired divergence affects how much extra work those stages need.

Primary key restrictions have different properties

The report distinguishes mutable ordinary columns from primary key components. A primary key does not undergo partial updates in the same way. A query restricting a clustering column and an ordinary column therefore does not necessarily have the same consistency hazard as an intersection over multiple mutable columns.

The candidate's changelog also includes a vector-search correction permitting clustering-key restrictions where filtering is not required. That is another example of why query shape matters. A restriction's location in the data model affects how it can be enforced, rather than every additional predicate having identical execution implications.

This is one reason a release candidate should be evaluated using the actual query shapes an application intends to issue. A single successful indexed lookup does not establish the behavior of intersections, static columns, or paging under replica divergence. The public test work names those distinctions explicitly.

A warning covers a separate filtering problem

The same candidate includes a guardrail warning for filtering queries over multiple mutable columns. CASSANDRA-19489 describes possible transient incorrect responses when such queries require coordinator reconciliation over unrepaired data. It also explicitly notes that the SAI-specific case has a correction through CASSANDRA-19018.

Those statements should be read together. The warning does not imply that the corrected SAI path remains identical to the general filtering path. Nor does the SAI fix establish that every other filtering mechanism has been corrected. The release contains both a specific implementation repair and a warning for a broader unresolved class.

The distinction affects how a caller interprets a warning. It reports a query shape and data-state risk rather than changing the requested consistency level into a guarantee the execution path cannot supply. The guardrail makes that limitation visible at the client boundary.

Index term limits can reject writes explicitly

Another RC1 change can fail a write when SAI refuses to index a term exceeding its configured maximum size. CASSANDRA-19493 explains the motivation as adding a hard failure option beyond a client warning for oversized text values.

That makes the relationship between storage and indexing an explicit policy decision. A caller may prefer a rejected mutation to a warning that its term cannot be indexed. The candidate provides that choice without making every index limit an implicit application convention.

For me, the useful story in RC1 is the work connecting new index capabilities to Cassandra's existing distributed semantics. Replica reconciliation, candidate retention, and client-visible guardrails determine what those capabilities mean under imperfect data agreement. The release candidate provides a concrete version against which that behavior can be evaluated before a final release.