September 1, 2026·4 min read·engineeringrqliterecoveryraftsnapshots

rqlite 10.3.0 ties fast restart to the correct snapshot

rqlite 10.3.0 adds snapshot index and term to its database fingerprint. The change prevents fast restart from assigning newer snapshot metadata to an older local database.

An unchanged database file can still be the wrong database state for a snapshot. rqlite 10.3.0, released on 1 September 2026, fixes a fast-restart path that failed to establish that relationship. The release also adds a console shortcut for querying table rows and updates SQLite. The restart repair is the change with the deepest consequence for how a node identifies the state it actually holds.

I value rqlite for putting SQLite behind Raft with a relatively small operational model. My year with it has made the boundary between local database state and replicated progress particularly interesting. A node's reported position has to describe the contents it can serve. A valid file checksum alone cannot establish that position.

A received snapshot was not necessarily restored

Issue #2747, reported by Rohan Padhye on 28 August, describes a safety problem found with Antithesis testing. A received snapshot could become visible in the snapshot store without being restored into the state machine. On a later restart, the old SQLite file could pass its fingerprint checks while startup adopted the newest stored snapshot's index.

The report observed missing committed rows on a read replica under network partitions and node kills. Local reads could return incomplete data even while the node advertised progress beyond the missing range. The immediate failure was not that SQLite could not read its file. The inconsistency lay between that file's state and the replication position assigned to it.

That distinction explains why file validation was insufficient. A checksum can confirm agreement with previously recorded bytes. If those bytes represent an earlier applied position, they remain earlier state after a successful checksum comparison. A second source of metadata cannot supply a newer position unless the implementation proves that the two sources correspond.

The fingerprint now identifies a snapshot position

The fingerprint implementation adds Index and Term to the recorded file description. Its validity predicate compares the file's modification time and size together with the expected snapshot index and term. CRC checking remains separate because calculating the checksum requires reading the file.

This changes the question answered during startup. The earlier file description established properties of a file. The new description also records which snapshot position that file represents. Comparing both kinds of information prevents a newer snapshot directory entry from silently redefining the meaning of an older file.

Index and term belong together in this comparison because the implementation uses both to identify the snapshot position. Checking only a byte count or a modification timestamp would leave the original relationship unproved. The correction explicitly carries the metadata needed for the decision instead of relying on the normal ordering of successful operations.

That is a useful design principle in recovery code. Related artifacts can be individually well formed while referring to different moments in a state transition. The durable representation needs enough information to detect that disagreement after the process that originally coordinated them is gone.

Startup declines an unproved shortcut

The store startup code reads the latest snapshot index and term before deciding whether the local database qualifies for fast restart. A fingerprint without the new fields does not establish the required correspondence. Startup falls back to restoration when that correspondence cannot be demonstrated.

This creates a visible upgrade consequence. An older fingerprint can still be decoded, but its missing position fields cannot justify the shortcut. The implementation comments describe one full restore on the first startup after upgrading such a node. That is an intentional cost of obtaining evidence the old fingerprint never recorded.

The tradeoff is specific to the fast path. The fix does not reject an old database merely because the marker uses an older format. It declines to skip the restoration that establishes a known relationship between snapshot state and local contents. Compatibility of the marker's encoding is different from sufficiency of its evidence.

My preference is for that conservative startup decision. The performance benefit of reusing a local file depends on knowing what it contains in replication terms. When the old metadata cannot answer that question, restoring supplies the missing state transition instead of assigning the file a position by convention.

The regression test separates persistence from application

The restart test added with the repair constructs a newer snapshot in the store without restoring it into the state machine. It then restarts the node and checks that startup performs restoration instead of taking the shortcut. This targets the relationship that ordinary successful snapshot tests keep synchronized.

That construction is more informative than restarting a healthy node repeatedly. A healthy sequence tends to leave the file and newest snapshot in agreement. The failure requires those artifacts to diverge, so the test deliberately creates that state. It then asserts the recovery decision that resolves the disagreement.

The fingerprint tests cover both directions of mismatch, along with a changed term and a legacy marker. That distinguishes a snapshot store ahead of the file from a file associated with a snapshot that never became visible. Both invalidate the same shortcut for different reasons.

The console shortcut remains an ordinary bounded query

The console change adds a schema-view action that opens the query tab and runs a SELECT with LIMIT 100. It quotes the selected table name through the console's identifier helper. This reduces the steps needed to inspect a table without introducing a separate data-access mechanism.

The changelog also records the SQLite dependency moving through 3.53.3 to 3.53.4 during this release's development. Those engine updates are distinct from the restart repair. Updating query execution code cannot by itself correct a node assigning metadata to the wrong local state.

With the new fingerprint, fast restart has an explicit correspondence check. A node can reuse its database when the file evidence and snapshot position agree. When they do not, startup restores the snapshot before treating its position as the database's applied state.