CockroachDB 20.1 beta 4 repairs concurrent schema writes

Adding a column online requires more than filling values into existing rows. Transactions writing while the backfill runs must also supply the values required by the changing schema. CockroachDB 20.1 beta 4 fixes a case where those concurrent writes could miss a column's default or computed value. The March 30 release notes describe the correction for inserts, updates and upserts while columns are being added or removed.

The release catalog identifies this build as a testing release dated March 30. Its changes remain part of the development of 20.1. That status matters when interpreting the behaviour as a versioned contract. A beta gives application authors concrete code to exercise, while its implementation can still change before production availability.

I value CockroachDB's serializable default, but online schema changes create another concurrency problem to solve. A transaction can interact with a table while a background operation is changing its representation. Preserving an application's logical schema requirements needs coordination between both activities, not merely successful execution of the ALTER statement that starts the change.

Backfill and foreground writes overlap

The online schema change documentation available with this beta describes a bridging strategy. Multiple schema versions can be in use while the system moves through discrete steps. A background job backfills or removes underlying data while the cluster continues serving reads and writes. Nodes eventually switch to the new schema after the required work completes.

That architecture makes concurrent writes part of the transition. A backfill accounts for the rows it processes, but foreground transactions can create or modify rows during the same period. Their writes must respect the requirements associated with the transition. A completed background scan alone cannot establish that another transaction produced every required value.

The beta fix addresses that overlap. It ensures affected columns receive their default or computed values during concurrent row writes. The release notes frame this as a correction to the column backfill process. It is a specific data correctness issue within online schema changes, rather than a reason to treat all schema operations as a single instantaneous event.

Changing a key can require several index rewrites

The ALTER PRIMARY KEY reference documents another 20.1 capability under development. A key change can rewrite multiple indexes. The old primary index becomes a secondary index, which preserves a path for queries using the old key columns. That behaviour matters when evaluating what a key change does to the table's physical access paths.

For an existing table with a suitable non-null id column, the operation has this form:

ALTER TABLE customer_accounts ALTER PRIMARY KEY USING COLUMNS (id);

This example assumes the table and candidate key already satisfy the operation's requirements. It is not a request to add a column and change the key inside the same arbitrary transaction. The documentation specifically separates primary-key changes from the transaction creating their table and disallows starting another primary-key change while one is underway.

I prefer describing such operations by the objects they rewrite. A short DDL command can initiate substantial distributed work. The old index's continued existence is also relevant to understanding the resulting schema, since changing the primary key does not mean the previous access path simply vanishes.

Uploading a file is an administrative operation

Beta 4 also requires the admin role for the new node-local upload command. The upload documentation explains that it uses a SQL connection to place a source file in a node's external I/O directory. By default that is the gateway node, and the file goes to only one node.

Those details prevent a misleading interpretation of the command. A SQL connection is the transport for an operation on a node's filesystem. It does not make the uploaded file a replicated SQL row. Specifying the destination node and retaining the returned node-local location are part of identifying where a subsequent operation will find the artifact.

The documentation requires external I/O to be enabled on that node and shows the uploaded file being used for import. Upload completion and import completion are consequently different milestones. One establishes that a file reached a particular destination, while the other establishes that the database processed that input through the requested import operation.

The administrative restriction matches that filesystem effect. Ordinary permission to write application rows is different from permission to place input files in the server's external I/O area. The beta's schema and upload changes therefore address separate forms of coordination, one between concurrent database operations and one between an authenticated SQL session and an operational file destination.