rqlite restores periodic Raft snapshot checks

A replicated database can keep answering requests while retaining an increasingly expensive history. rqlite 5.2.0 fixes a snapshotting defect that allowed its Raft log to grow without bound. The April 11 announcement says the bug prevented snapshots and subsequent log truncation. The release's changelog identifies the cause as a zero interval passed to the Raft subsystem.

That failure connects storage growth with recovery work. The tagged changelog says a sufficiently large log could cause very long startup times. Continued service during normal operation therefore did not establish that the node's retained history remained manageable. The maintenance mechanism needed to run before the accumulated history became a restart problem.

I value rqlite for providing a modest amount of distributed-systems machinery around SQLite. That design still needs the maintenance work that makes a replicated log usable over time. A small deployment should not have to retain every historical command merely because ordinary writes continue succeeding.

A snapshot represents applied state

The store implementation in 5.2.0 creates a file snapshot store when opening the node. Its comment states the purpose directly, allowing Raft to truncate the log. This is separate from opening the SQLite database and the persistent Raft log store. The node has related state representations with different roles.

The snapshot callback collects database contents and serializes cluster metadata. Its persistence method writes a database-size field, the database bytes and then the metadata to the snapshot sink. The restore callback reads those components and rebuilds the database state. The snapshot therefore represents application state that the log has already produced, with the information required to restore it.

This explains why truncation needs a successful snapshot. Removing log entries without a replacement representation would discard information needed to reconstruct state. A completed snapshot provides that replacement for the covered portion of history. Subsequent log entries can describe later work relative to the saved state.

The implementation supports restoring both an on-disk database and an in-memory database. For the latter, it writes the snapshot's database bytes to a temporary file and loads the in-memory database from that file. An in-memory SQLite configuration consequently does not remove the replicated system's need for persistent recovery state. The snapshot and log remain part of its recovery mechanism.

The snapshot also includes metadata, which distinguishes it from treating the operation as a copy of one SQLite table. Recovery needs the node's represented application state, including the metadata the store serializes alongside database contents. The persistence layout in this version makes that boundary explicit. A snapshot's role follows the state machine it represents.

Persistence also has an error path. If writing the snapshot fails, the rqlite implementation cancels the sink and returns the error. Raft checks the persistence result before advancing its recorded snapshot state. These checks matter because beginning a snapshot is not equivalent to completing a recoverable representation of the state.

The store keeps a snapshot counter and a separate restore counter. Their presence reflects two different operations in the implementation. Capturing state prepares a representation for later use, while restoring consumes that representation to rebuild the local database and metadata. The periodic-check fix enables the capture path to occur under its configured conditions.

The interval schedules a decision

The release adds a launch-time option for the Raft snapshot-check interval and defaults it to 30 seconds. The server entry point at this tag calls that option -raft-snap-int. It separately defines -raft-snap, whose default is 8192 outstanding log entries. These are different controls.

An illustrative launch can state both defaults explicitly. It starts a single node with the supplied local data directory and does not configure a multi-node topology.

rqlited -raft-snap 8192 -raft-snap-int 30s ~/node-1

The interval determines when Raft checks whether a snapshot is appropriate. The threshold determines how much log growth makes the check eligible to proceed. A 30-second interval does not promise a new snapshot every 30 seconds. With insufficient new history, the periodic check has no snapshot to trigger through that threshold rule.

The entry threshold is also not a byte limit. Commands can have different encoded sizes, so 8192 entries do not describe a fixed amount of storage. The setting counts log progress, while snapshot size follows the represented state. Describing either control as an exact cap on disk consumption would go beyond its implementation.

In the corrected store configuration, nonzero snapshot interval and threshold values override the Raft defaults. Other timing values use similar conditional overrides. This matters because an application's zero-valued field need not express a useful runtime default for its dependency. Passing configuration across that boundary requires preserving the dependency's intended behavior.

Raft checks growth before taking the snapshot

The Raft 1.1.1 snapshot implementation shows the periodic path. A timer wakes the snapshot loop, which calls shouldSnapshot(). That method obtains the last snapshot index and last log index, subtracts them and compares the difference with SnapshotThreshold. Only a qualifying result calls the snapshot procedure.

The timer path explains the effect of a missing periodic interval. Having a threshold configured does not make the decision run by itself. The process needs an occasion to evaluate the threshold. The rqlite changelog identifies the zero interval as the reason snapshotting had not occurred, despite the existence of the surrounding snapshot machinery.

The dependency's snapshot procedure asks the state machine for a snapshot, collects the committed configuration and creates a sink. It persists the state and checks completion before recording the new snapshot position. The log compaction step follows those successful operations. An error at an earlier stage therefore does not justify assuming the covered log entries have been replaced by a usable snapshot.

This ordering is important for interpreting the release. The fix restores the periodic opportunity to take a snapshot and compact covered history. It does not turn the interval into unconditional deletion. A valid state snapshot remains part of the sequence that precedes compaction.

Compaction retains a useful log tail

The Raft implementation does not simply delete the entire log after snapshot creation. Its compaction code takes both the snapshot position and a trailing-log retention setting into account. It removes only the eligible earlier range. Entries after the snapshot cannot be removed through that snapshot, and a retained tail can remain even within the covered range.

This gives log truncation a more precise meaning than reducing the file to zero entries. The successful snapshot establishes how much history can be represented by saved state, while retention determines part of what remains available as log entries. Both influence the post-snapshot log. The rqlite release fixes the missing maintenance path without promising that no log remains afterward.

It also helps distinguish logical compaction from a generic filesystem cleanup. The eligible range follows Raft indexes and snapshot coverage. An operator cannot infer that an arbitrary old-looking file is dispensable merely because snapshotting exists. The running system's storage interfaces carry the information needed to remove the correct history.

I prefer understanding the fix through those dependencies. The check must run, the threshold must be met, and snapshot persistence must complete before eligible history is compacted. This is an operational mechanism with observable stages, even in a database whose process model remains small.

Startup cost depends on retained history

The changelog's warning about startup time follows from having a large retained log. Restoring state from a suitable snapshot avoids treating all prior command history as equally necessary replay work. Newer entries still describe work after the snapshot. With periodic snapshotting absent, the amount of retained command history can continue increasing as the node processes changes.

The release does not quantify a universal startup improvement or give a maximum recovery duration. Workloads differ in command history and database state, and snapshot creation itself performs real database and metadata work. The specific correction is restoring the maintenance path that prevents the described unbounded growth, with controls exposed for its periodic check.

For applications that choose rqlite to avoid disproportionate database operations, that correction is material. A bounded voting topology and a single server binary do not make accumulated history irrelevant. Version 5.2.0 provides the documented 30-second check default and launch-time interval control, allowing the threshold-driven snapshot process to run and covered log history to become eligible for truncation.