rqlite 9.0.0 gives database changes a durable HTTP delivery path

A replicated database can agree on a write while an external consumer still knows nothing about it. rqlite 9.0.0, released on 15 September, adds change data capture to connect those two systems. It captures changes in SQLite and delivers JSON to a configured HTTP endpoint. The difficult part is preserving delivery work across restarts and leadership changes.

I value rqlite's restrained operational model. CDC is useful when relational state needs to drive another system without requiring applications to poll tables for changes. Its design also makes the external boundary explicit. Committing a database operation and receiving an HTTP acknowledgement remain separate events.

Capture occurs while applying the replicated operation

The released store implementation registers a SQLite pre-update hook and a commit hook when CDC activates. Before processing a Raft log entry, it resets the CDC streamer with that entry's index. That index relates captured changes to the replicated operation being applied.

These hooks do different jobs. The pre-update hook receives the row-change information, while the commit hook participates in handing committed changes to the stream. Hook registration stays in the application path, where the store records whether it has already registered them.

Database replacement requires special handling. The store explicitly clears its CDC registration state after swapping in a database or restoring one. A subsequent application registers the hooks again. Preserving database bytes does not preserve every callback attached to an earlier database instance.

This is a useful detail for understanding the implementation. CDC is not simply another HTTP handler around a query. It depends on the lifetime of SQLite connections and on the store's application of replicated commands. Recovery code has to restore that relationship when it replaces the underlying database.

Enabling CDC identifies a destination

The configuration parser accepts an endpoint URL, a JSON configuration file, or the special value stdout. A URL gets the default configuration with that destination. A file permits additional settings such as filtering, batching and retry behavior.

A small configuration file can identify the stream and select a table-name expression. This example restricts capture to a table named orders and leaves timing defaults to the parser.

{
  "endpoint": "https://consumer.example/events",
  "service_id": "order-state",
  "table_filter": "^orders$"
}

The server's -cdc-config option selects that file. The optional service identifier lets a consumer distinguish streams from multiple rqlite systems. It is an application-supplied identity, so choosing a stable value is part of designing the receiver's records.

The configuration also offers row-IDs-only capture. That can reduce the data in delivered events when a consumer needs identities rather than before-and-after column values. The choice changes what information the receiver can recover from the event itself.

The server configuration checks reject CDC on non-voting nodes. This is a concrete release constraint. A deployment should not assume that every role capable of holding database data can also enable this CDC service.

Queue persistence separates delivery from application

The CDC service batches incoming event groups, marshals them into an envelope, compresses the result and enqueues it in a persistent FIFO. The queue entry carries the highest index represented in the batch. This groups work for transport without losing its replication position.

Batching runs for followers as well as leaders. Each enabled service can therefore retain work locally before it becomes responsible for transmission. Leadership determines which loop delivers events to the endpoint, while queueing records the changes that service receives.

The persistence boundary matters because the in-memory batcher and disk-backed FIFO have different lifetimes. The source includes snapshot synchronization that flushes through the batching path. A database snapshot must not make the CDC service depend only on commands that the replication log is about to discard.

The release changelog records this snapshot integration explicitly. It also records restart tests and tests for different event types. Those changes address the relationship between database recovery and event delivery, beyond the visible JSON format.

A separate queue gives delivery its own storage lifecycle. Retaining unsent work there allows the external endpoint's availability to differ from the database's normal application progress. The resulting queue can require capacity even when the database itself is operating normally.

Only the leader transmits

On a leadership change, the service stops the previous role's loop and starts the appropriate leader or follower loop. The leader consumes queued entries and constructs HTTP POST requests with JSON content. Followers process progress observations instead of independently delivering the same stream.

The transmission loop checks each queue entry against its current high watermark. Entries at or below that position are already known as delivered and can be skipped. After a successful send, the service advances its watermark to that entry's index.

A separate periodic loop broadcasts the watermark to other nodes. It broadcasts an unchanged nonzero value as well, so another node can learn progress even when no new batch arrives. Locally, it only repeats pruning when the watermark advances.

Followers use those updates to delete queue entries through the delivered position. That is how a node can retain a local copy of delivery work while avoiding routine duplicate transmission after it becomes leader. Its queue and the cluster's delivery information inform the next delivery decision.

This sequence also explains why acknowledgements and leader changes matter to consumers. There is an interval between a receiver accepting a batch and all nodes learning that result. A replacement leader can have insufficient evidence that the earlier batch was accepted.

The receiver controls what an acknowledgement means

The leader loop treats HTTP 200 and 202 as success. An HTTP 204 response is not included in that success condition. A generic webhook implementation that returns any successful status code therefore needs to match the sender's actual acceptance rule.

The difference between 200 and 202 is also relevant to downstream processing. The sender knows that it received an accepted response. It does not inspect whether the receiver has finished an asynchronous database update or message publication. The receiver must decide when it has made the event safe enough to acknowledge.

For a consumer maintaining a derived database, a useful design is to record the event's replication position with its applied result. That is an application design judgment, rather than an automatic property supplied by HTTP. It gives duplicate handling information that survives a consumer restart.

My preference is to make that durable consumer decision before acknowledging a batch. A response sent before the receiver has preserved either the work or its completed effect can let rqlite advance delivery progress while the receiver still has only volatile state.

Retry policy changes the delivery contract

The configuration defaults to retrying without a finite maximum. It also supports a maximum retry count, after which failed events can be dropped. These are different operating policies, and a statement about eventual delivery needs to identify which policy is in use.

The defaults specify a five-second transmit timeout and a one-second minimum backoff, with linear retry policy. Exponential retry is another configured option. In the transmission loop, exponential backoff doubles the retry delay and caps it at the configured maximum.

Batch defaults are ten events and a 200-millisecond delay. Those settings trade request frequency against waiting for a fuller batch. They do not make the endpoint's processing time disappear, and retrying a blocked destination can leave unsent work in persistent storage.

The source comments contain an older description of a 100-millisecond minimum backoff, but the actual default constant is one second. The parser assigns that constant when the JSON field is zero. Configuration behavior should follow the implemented constant and defaulting code.

Delivery identity belongs to an event group

Philip O'Toole's announcement describes at-least-once delivery and explains the duplicate risk around restarts and leadership changes. Its payload example groups row events under a Raft index. A consumer must preserve that grouping when it uses an index to recognize repeated work.

An index should also be interpreted within the source system. The optional service identifier is useful when one receiver accepts independent streams. Treating a numeric index from unrelated clusters as one shared sequence would discard information needed to distinguish their events.

For an existing installation, the changelog recommends upgrading through 8.43.4 and backing up before moving to this release. It supports rolling upgrades and describes downgrading to the 8.x series as possible. Those compatibility statements accompany a feature whose effect continues beyond the database process, through the receiver's acknowledgement policy and durable handling of delivered groups.