CockroachDB vector indexing gains online backfills at general availability

A vector index that requires a populated table to stop accepting writes is difficult to introduce into an existing application. Similarity search may be the new feature, but the table already has users and ongoing transactions. The November 3 release of CockroachDB 25.4 makes vector indexing generally available and removes that particular adoption constraint through online backfills.

The change concerns building the index over existing rows while writes continue. It is separate from the ability to update an index after construction, and separate again from the quality of approximate search results. General availability brings those operational questions into sharper focus because an application needs a usable path from its current schema to the new index.

I find the integration with transactional data more interesting than adding another place to store embeddings. CockroachDB already makes consistency and distributed placement explicit parts of its model. A vector access path is useful when it fits that model and can be added without suspending the table's ordinary workload.

Approximate search narrows the candidate set

The vector index guide describes a hierarchy of partitions built using k-means clustering. Similar vectors are grouped together. A search explores partitions near the query vector, evaluates candidates using the configured distance metric and returns nearby results without comparing the query with every stored vector.

This is approximate nearest neighbor search. The index does not promise that it examines every candidate that an exhaustive distance calculation would consider. Query latency and search accuracy are therefore related through how much of the structure is explored. Increasing the work can improve the chance of finding the nearest vectors, while consuming additional CPU and increasing read latency.

The distinction between a vector value and a vector index matters here. The VECTOR type reference documents fixed-dimensional floating-point values and distance operators. Those operations are useful even without an approximate index. Indexing changes the access method for a similarity query, rather than creating the underlying ability to represent or compare vectors.

The type declaration can enforce dimensionality. A column declared VECTOR(3) accepts three-dimensional values, while an application using a larger embedding model declares its corresponding dimension. An index cannot reconcile vectors from incompatible representations merely because both are arrays of numbers. The database's dimension constraint addresses shape, while the application's choice of embeddings determines what proximity means.

The distributed structure builds on the key-value layer

Cockroach Labs' June explanation of C-SPANN describes storing index partitions in the database's key-value layer. Partition data can consequently participate in the range splitting and rebalancing machinery used for other data. A logical partition in the similarity tree is different from a physical range, even though both can split.

The same explanation describes incremental partition maintenance. Growing partitions can split, shrinking partitions can merge, and vectors can move as centroids change. These operations maintain the search structure as data changes. They are different from rebuilding the whole index whenever the application inserts another group of vectors.

That architectural fit is the relevant distribution claim. A database range determines physical placement, while a vector partition groups nearby points for search. Conflating the two would suggest that a storage rebalance necessarily changes the similarity grouping, or that a clustering change necessarily corresponds to a new storage node. The two structures solve different problems and interact through the stored representation.

I prefer that separation because it makes the cost easier to reason about. Distribution supplies placement and resilience, while approximate search decides how many candidates to inspect. Neither property establishes a workload-independent latency figure, and the release does not remove the need to understand the access path used by a particular query.

Online backfill records changes in a different index form

The online backfill implementation identifies the previous restriction directly. Mutations of values in a vector index undergoing backfill were prohibited, effectively taking the table offline while the index was built. The patch reenables those mutations and changes how concurrent updates are captured.

Instead of using another vector index as the temporary structure, the implementation uses a forward index. Its keys contain the vector index's prefix columns and the row's primary key, while the vector itself is stored as payload. This records changes in a form that can be merged into the growing vector structure.

The merger then reencodes vector entries read from that temporary index. This distinction is necessary because a vector entry's destination depends on the vector structure, rather than just copying the same forward-index key into another location. Supporting concurrent writes therefore requires a merge operation aware of the vector representation.

Another change in the same patch handles primary-key lookup during schema transitions. Fetching full vectors must use a primary-key-encoded index whose columns match the key embedded in the vector entry. The implementation searches the available indexes for that match instead of assuming the currently designated primary index is always the right one. This illustrates why online construction is schema-change work as well as search-algorithm work. A candidate has to lead back to its full vector through the correct row encoding while the schema is changing.

The patch also describes a subtle deletion consequence. Forward-index tombstones do not retain values, so vectors deleted during backfill are not directly deleted from the vector structure by that merge. Reads still join candidates to the primary index, preventing deleted rows from being returned. A large number of such deletions can nevertheless affect search accuracy by leaving obsolete candidates in the approximate structure.

That is a useful example of separate correctness properties. Excluding a deleted row from SQL results does not establish that every approximate-search candidate slot was spent on a live vector. The implementation comment explicitly distinguishes result correctness from search accuracy during this transition, which is more informative than treating online construction as a single yes-or-no property.

There is also a documentation discrepancy to resolve. The vector guide retains wording that building an index on a non-empty table blocks writes. The 25.4 beta release note and the implementation explicitly remove that behavior, and the final release highlights online backfills. For this release, those change records establish the new behavior more directly than the guide's older setup paragraph.

Query shape still determines index eligibility

The guide supports prefix columns before the vector column. These let a query narrow the search space to a particular category or owner before comparing vector distances. The query must constrain the prefix columns appropriately for the vector index to be used.

For a small illustration, this schema declares an L2 vector index prefixed by a customer identifier. The matching query supplies that identifier and orders by the L2 distance operator:

CREATE TABLE items (
    id INT PRIMARY KEY,
    customer_id INT NOT NULL,
    embedding VECTOR(3),
    VECTOR INDEX (customer_id, embedding)
);

SELECT id
FROM items
WHERE customer_id = 7
ORDER BY embedding <-> '[1.0, 0.0, 0.0]'
LIMIT 10;

The three-dimensional value keeps the example readable. Its purpose is to show the relationship between the prefix predicate and the ordering expression, rather than model an actual embedding workload. A search missing the prefix restriction does not gain the same access path simply because it references the indexed vector column.

Operator class selection matters as well. The guide lists vector_l2_ops as the default, vector_cosine_ops for cosine distance and vector_ip_ops for negative inner product. The operator in the query must correspond to the index's distance metric for acceleration. Supporting several distance operators does not make one index interchangeable across all of them.

Search effort remains an explicit tradeoff

The documented vector_search_beam_size session setting controls how many partitions are considered at each level. Larger search beams examine more of the index. Partition size provides another tradeoff, with larger partitions supplying more candidates and reducing the frequency of structural splits and merges.

Those choices influence both read and write costs. More candidates can improve accuracy, but require more work during search. Larger partitions can improve insertion behavior by reducing maintenance frequency, while increasing the amount of data inspected per partition. The relevant setting depends on the distribution of the actual vectors and the query's filtering behavior.

The documented limitations also distinguish bulk import from index construction. IMPORT INTO is unsupported on a table that already has a vector index. The documented route imports vectors first and creates the index afterward. Online backfill makes that ordering more practical for an active table, while leaving the import restriction as a separate property of the ingestion path.