CockroachDB 1.1.3 cleans up data left by failed restores

A restore that fails can have already written data successfully. CockroachDB 1.1.3 fixes a case where those writes remained permanently allocated after a failed or cancelled enterprise restore, without a table descriptor through which anything could see or delete them. Recording the job as failed did not complete the cleanup of the work it had performed.

The November 27 release notes include cleanup of partially restored data and a separate fix for resuming restores that skip foreign keys. Both concern the lifecycle of a long-running operation. Its behavior after failure or interruption is part of its correctness, alongside the rows produced by a successful run.

I regard recovery tooling as part of the database's operational contract. Restoring a backup must account for incomplete work because interruption is a plausible condition during recovery. A failed job should leave a state the system can manage, rather than consume storage through unreachable data.

Data existed without the descriptor needed to manage it

The cleanup backport, merged October 31, describes the old behavior. Data committed before the restore failed or was cancelled became orphaned. With no table descriptor, ordinary table operations could neither inspect it nor delete it, so it continued to occupy space.

This is different from a table visible to a user but containing only part of the expected rows. In that case, the table remains an object an administrator can address. Here the missing descriptor removed the normal relationship between stored data and a manageable SQL object.

The distinction explains why marking the restore failed was insufficient. The job record described the administrative outcome, but data written during earlier stages had its own persistence. A lifecycle implementation has to connect those states explicitly if failure should reclaim the partially completed restore.

The source describes the correction for failed or cancelled operations. It does not supply a general claim that installing 1.1.3 automatically identifies and reclaims every historical orphan from every earlier failure. The verified change is the new cleanup behavior associated with the restore's failure path.

Job failure gains an operation-specific callback

The implementation adds optional callbacks so a particular job type can define an onFail method. It also passes the job's transaction into the update and failure callback. The stated purpose is to let the system fail the job and clean up transactionally.

That mechanism puts the operation-specific knowledge beside the general job lifecycle. The job system can record failure, while the restore-specific callback knows which partial state must be removed. Providing the transaction connects those actions instead of treating cleanup as unrelated work after the status update.

The change is therefore more precise than a periodic sweep for unidentified files. It extends the job failure path so the restore can participate in making its own incomplete work reclaimable. The pull request explicitly ties the callback and transaction changes to cleanup of committed restore data.

I find this useful as an example of cancellation semantics. A cancellation request is only the beginning of the operation's response. The implementation must decide what happens to already completed stages and record an outcome consistent with that decision. This fix supplies that missing relationship for restore failure and cancellation.

Resuming must reconstruct the same descriptors

A separate restore backport, merged November 21, addresses jobs resumed after a pause or node failure. A resumed restore regenerated SQL descriptors through a different path from the initial invocation. When the user had requested skipping foreign keys, the descriptor mutation did not happen consistently across those paths.

The correction moves that mutation downstream so both initial and resumed execution modify the index descriptors in the same way. It preserves the user's selected behavior across interruption. Resuming the job should not reinterpret the restore options merely because execution entered through another path.

This is a different lifecycle requirement from cleanup. Failure needs to dispose of incomplete work, while resumption needs to continue under the same intended definition. The two fixes share restore machinery but solve separate state-transition problems.

The explicit foreign-key condition also limits the claim. The release notes identify resumed jobs that skip foreign keys. They do not say that every resumed restore previously failed, or that the fix changes the meaning of every restore option. The implementation description supplies the affected descriptor transformation.

Query fixes accompany the operational repairs

Version 1.1.3 also corrects results for some queries with IN constraints, index encoding for STORING columns, and multi-column PARTITION BY with window functions. Other fixes address panics in several SQL and tracing paths. These are application-visible corrections independent of enterprise restore usage.

The release adds historical-query support to SHOW TRACE FOR SELECT and avoids overloading the system during cleanup of large transactions. Those entries reinforce the need to distinguish normal execution, observation, and cleanup as separate paths with their own resource behavior.

For restores, the central changes are the failure callback and consistent descriptor reconstruction. A failed or cancelled job can clean up its partial data through the job transaction, while an interrupted job resumed with foreign keys skipped now applies the same descriptor mutation as its original invocation.