October 13, 2024·6 min read·engineeringrqliteraftconsistencyreads

rqlite adds reads that verify leadership without logging SQL

rqlite 8.32.0 adds a linearizable query mode that checks leadership after reading. The implementation keeps term validation separate from the existing strong read through Raft.

A node can believe it is the leader after the rest of a cluster has elected someone else. That makes a local leadership flag insufficient evidence for a read that promises to respect completed writes. rqlite 8.32.0, released on October 13, adds a linearizable read mode built around checking leadership without putting the query itself through the replicated log.

rqlite already has a strong read path that submits a query command through Raft. The new mode aims to remove that logged query operation while retaining an explicit check involving the consensus layer. Its implementation shows the distinction between reading the local database and establishing that the result may be returned under the requested consistency policy.

I value rqlite's request-level consistency controls because they make the cost of an operation a deliberate choice. A read that can avoid replicated SQL while still verifying leadership is a useful addition to that model. The interesting question is how the implementation connects the data it reads to the leader's authority during the request.

The argument included a leader returning to office

The source links an October raft-dev discussion started by Philip O'Toole on October 3. The early messages ask whether a leadership check and a database read can safely happen as separate steps. They also consider a process being delayed long enough to lose leadership and later regain it. In that case, a successful later check alone would not establish that an earlier read belongs to the same leadership term.

By October 8, the discussion identifies term continuity as part of the check. This gives useful historical context for the released implementation. The code records the current term before processing and compares it again after verifying leadership. It does not treat the identity of the node as sufficient continuity across elections.

The distinction is practical even though the scenario is unusual. A process can remain the same process while its authority changes. The local database may contain a result read under an earlier period of leadership. A later successful leadership verification cannot make that earlier result current merely because the same machine is leader again.

The release changelog associates the new mode with several implementation changes. The feature concerns the ordering of checks around query execution, not simply another spelling for an existing consistency setting. Its source is therefore a better guide to the new behaviour than the name by itself.

A new parameter selects a separate path

The query parameter parser accepts linearizable as a consistency level. Its fallback remains weak. An application must select the new level explicitly, and existing requests do not acquire it merely because the server has upgraded.

The request has the same query endpoint and SQL representation as other reads. The new parameter expresses the additional consistency requirement.

curl --get 'http://localhost:4001/db/query' \
  --data-urlencode 'q=SELECT 1' \
  --data-urlencode 'level=linearizable'

The store query method checks that the node is open and performs the existing PRAGMA check. For linearizable consistency, it also requires local leadership and readiness before proceeding. A node that cannot meet those prerequisites returns an error through the existing store interface.

Readiness has its own definition in this version. The store must be open, its registered readiness channels must be complete, and it must know a leader. The linearizable branch adds its explicit local-leader test alongside that existing readiness predicate. These checks establish conditions for entering execution, while the later verification decides whether an otherwise successful query can finish successfully.

Verification follows database execution

Near the beginning of query processing, the store records the current Raft term. A deferred function runs when the method is about to return. If the query otherwise succeeded and its selected level is linearizable, that function verifies leadership. It then checks whether the term still equals the recorded term. A changed term produces the store's stale-read error.

That deferred placement is the defining detail. The method can call the local database and collect rows before completing the leadership checks required for the response. Reading rows internally is therefore not the same as authorizing a successful result for the caller. A failure in the final checks turns the store call into an error even when SQLite produced data.

The leadership helper delegates to Raft's VerifyLeader operation and waits for its future. If Raft reports that the node is no longer leader or has lost leadership, the wrapper maps that to rqlite's not-leader error. Other verification failures receive additional context in the returned error. The helper also records successful and failed verifications separately.

Term comparison follows a successful verification. The code obtains the term through Raft statistics and parses its numeric value. If the term is missing or cannot be parsed, the helper returns zero. That implementation detail belongs to how this release obtains the value, rather than to a new application-visible term parameter. Clients request consistency, while the store performs the term bookkeeping.

I find the distinction between collected data and accepted data useful here. The store's result is a combination of local SQL execution and the outcome of the surrounding checks. A client should treat an error from that operation as authoritative, regardless of whether the server briefly held a row buffer that looked usable.

Strong reads still submit a command

The strong branch remains visible beside the new code. It serializes the query request into a command and invokes Raft's apply method. The response comes from that applied command's result. The linearizable branch checks its prerequisites and then reaches the ordinary local database query call, with deferred leadership and term checks still pending.

This difference explains the intended reduction in overhead without requiring a benchmark claim. The new path avoids serializing and applying the query as a replicated command. It still performs consensus-related leadership verification. A comparison that describes it as a read with no cluster coordination would omit the very mechanism that distinguishes it from a weak leader read.

Weak consistency still checks the local leadership state before reading, but it does not select the new deferred verification. The none level retains its stale-read handling. Automatic consistency still resolves according to node role. The additional mode therefore provides another explicit path within the request policy, and it does not silently strengthen every existing read.

The query implementation's ordering also distinguishes this mode from a general clock-based lease. The code compares term values and asks Raft to verify leadership. Its visible acceptance condition does not consist of assuming a lease remains valid for a configured local time window. Describing the implemented checks directly avoids attributing a different algorithm to the release.

Endpoint selection also matters in this version. The deferred verification described here belongs to the store's query method. Its separate unified request method has another control flow and does not contain the same linearizable branch. A client abstraction that routes all SQL through the unified endpoint should not assume that recognising the parameter in the shared parser establishes identical enforcement in both store methods.

The response must reflect the final check

The HTTP query handler receives the store's results and error separately. When the store returns an error, the handler records the error instead of selecting the successful query-row response. That matters for a mode whose final validation occurs after local database execution. The surrounding API has a way to reject the result at that point.

The store tests include a single-node linearizable query after creating and inserting a row. They check the column names and returned values. That establishes the basic successful path, while the source makes the additional rejection conditions explicit. A successful ordinary read test alone would not explain the term-change check or the separate leadership-verification future.

The new mode is most clearly understood through those control-flow differences. It requests a local query surrounded by explicit leadership validation, while the existing strong mode applies a query command through Raft. Both the request parameter and the response error handling matter to that distinction. An application selecting the new level needs to preserve its errors through its own database abstraction so that a rejected read does not become an accepted result elsewhere.