CockroachDB scan continuation could skip isolated keys
Advisory A-74736 concerns incomplete query results when a scan crosses ranges and mixes isolated keys with enough data. The failure came from metadata describing unfinished requests.
A database request can stop after returning a bounded batch without finishing all its work. The caller then needs an accurate description of what remains. If that description omits an unfinished request, the next batch can continue successfully while silently skipping data.
The February 7 CockroachDB advisory A-74736 describes this kind of failure in primary and unique index scans. The affected plan retrieves at least 10 MB, crosses multiple ranges, and scans isolated index keys, such as keys derived from an IN condition. CockroachDB versions 21.2.0 through 21.2.4 can return incomplete results. Version 21.2.5 contains the correction.
I accept the visible coordination costs of CockroachDB because its consistency model is useful. This advisory concerns another necessary part of correct distributed execution. The SQL layer needs the storage layer to describe unfinished work accurately, even when every individual key read is otherwise valid.
The report first looked like a large-query problem
The original issue 74736, filed January 12, describes a query selecting about 140,000 known identifiers. The single query returned about 28,000 rows. Dividing the identifiers into smaller batches returned the expected collection. The report also supplied a query plan and an obfuscated schema.
That observation was useful without establishing a general identifier-count threshold. An IN list is an SQL description of requested values. Its storage work depends on the plan and the location of those values. The eventual advisory defines the conditions in terms of data volume, range crossings, and isolated keys, not one universally unsafe list length.
The issue reports the same behavior through several clients. That made a client-specific row-fetching mistake a less convincing explanation. The query plan also showed fewer rows arriving from KV than the reporter expected. The investigation therefore had evidence to examine the server's read path rather than treating the discrepancy as a presentation problem.
The report remains someone else's observation, not an account of my own deployment. Its value is the comparison between logically equivalent retrieval attempts and the execution information attached to them. That comparison gave the maintainers a way to ask which ranges and requests each form actually visited.
Isolated keys and scans used different request forms
The discussion distinguishes point reads from scans. A point read asks for a particular key, while a scan covers a span. A query containing many identifiers can cause both forms to appear in the work sent through the range machinery. The SQL expression does not imply that every requested identifier travels through one identical storage operation.
Maintainers compared traces from the incomplete retrieval and a count query that visited more ranges. They observed that omitted ranges involved point requests. After an initial response needed continuation, later work still included scans while missing some of the point reads. This narrowed the investigation to how unfinished work was represented between batches.
The distinction matters because two request forms can share a batch while relying on different completion metadata. A caller that assumes every unfinished request carries the same marker needs the lower layer to honor that contract for both. Returning partial data without the marker can make the caller infer completion where none occurred.
The metadata contract was incomplete
A January 21 comment in the issue identifies the mismatch. The KV batch fetcher expected a resume span for every request that had not been satisfied. The distribution layer filled that field for scans, but did not do so for the point reads in question. That explains why scans could continue after the first fetch while some isolated keys disappeared from the remaining work.
The advisory summarizes the repair as correcting metadata returned by the KV layer when isolated key spans were not scanned. This is more specific than saying the query engine became better at handling large results. The actual values were not necessarily missing from storage. The continuation information failed to keep some requested reads in the pending set.
The issue later supplies a small multi-range reproduction mixing a range condition with one isolated key. With one execution arrangement, the count includes the extra key. With another, the count is one row smaller. That reduction connects the large original report to the same request-shape boundary without requiring the original data set.
I find that reduction informative because it preserves the mechanism while removing the misleading scale of the first symptom. The important property is not that an application has an unusually long statement. It is that bounded retrieval must preserve every unsatisfied request across the continuation boundary.
Query correctness and index corruption require different responses
The advisory describes queries omitting rows and directs affected users to update to 21.2.5 or later. It does not prescribe rebuilding indexes. That differs from an index-maintenance defect where the persistent index contents themselves have become inconsistent with the table.
Here the read process skips work under the documented conditions. Reconstructing an index is not the remedy the advisory supplies for that metadata failure. The distinction prevents an expensive maintenance action from substituting for the implementation update that addresses the read path.
The advisory also limits exposure to specific 21.2 maintenance versions. Its statement that the case is rare does not change the result when a query reaches those conditions. An incomplete successful query can look ordinary to an application unless the application has an independent expectation about the result set.
The report's small-batch comparison helped expose the issue, but it should not be converted into a general claim that a particular batch size is a complete fix. Different row widths and range layouts change the work performed. The release correction repairs the unfinished-request metadata, which is the property the caller needs regardless of the application's chosen identifier grouping.
A correct continuation describes everything still owed by the previous batch. Version 21.2.5 changes that description for the affected isolated keys, so later retrieval can include reads that earlier versions incorrectly treated as finished. That is the concrete change behind the advisory's recommendation to update.