rqlite replaces RANDOM calls before statement replication

Replicating a SQL statement does not guarantee replicating the same result when that statement asks each database to make an independent random choice. rqlite 7.7.0, released on September 28, adds support for RANDOM() through statement rewriting. The receiving node replaces the random call with a concrete integer before the write enters replication. Other nodes then execute the same literal value.

The tagged release documentation identifies the underlying problem as statement-based replication. Each node reads SQL from the Raft log and applies it to its local SQLite database. If each application of the statement evaluates randomness independently, agreement on the log does not produce agreement on stored values.

I value rqlite's use of a statement log because its relationship to the local SQL engine is understandable. This release makes one necessary consequence explicit. Any choice that affects replicated state needs to be represented consistently in the work all nodes apply. Making the random value part of the statement supplies that consistency for the supported rewriting path.

The chosen value becomes replicated input

Consider a table with a column intended to receive one random value during an insert. Without rewriting, the statement asks each SQLite engine to evaluate a function. With rewriting, the SQL reaching the log already contains the selected value.

INSERT INTO sample(value) VALUES(RANDOM());

-- Illustrative rewritten statement, with an example chosen value:
INSERT INTO sample(value) VALUES(-428017);

The particular number above is illustrative. The important property is that every replica receives the same integer literal. Replaying that statement later does not need to reconstruct a previous random-generator state or reproduce the original random choice.

This changes where nondeterminism occurs. The receiving node makes the choice once during statement preparation for replication. The replicated operation then carries the outcome. Agreement concerns the resulting statement, and the replicas no longer independently choose the value for that call.

The mechanism does not require the SQL engine to become deterministic in every context. It removes a supported nondeterministic operation from the statement that replicas will evaluate. That is a narrower intervention with a concrete effect on the replicated input.

The implementation parses SQL before changing it

The rewrite code in 7.7.0 parses each statement through the project's SQL parser. It constructs a rewriter with random rewriting enabled and walks the parsed statement. Only after a successful transformation does it replace the original SQL with the statement's serialized form.

Parsing matters because a function call is a syntactic construct. The text RANDOM() inside a quoted string is data, while the same characters used as a call describe evaluation. A syntax-aware transformation can operate on the latter without treating every matching byte sequence as executable SQL.

The code deliberately leaves statements unchanged when parsing fails. Its comment says SQLite should then report the syntax error, since the engine's message is likely to be clearer. It also leaves the statement unchanged if rewriting fails or makes no change. This avoids making the auxiliary parser the only authority on every statement the engine might accept.

That fallback has a corresponding limit. An unchanged statement still has whatever semantics SQLite gives it. The rewrite stage cannot be read as a proof that every possible SQL input has had every nondeterministic effect removed. The implementation applies a specific transformation where its parser and rewriter can perform it.

The syntax tree controls which calls are evaluated

The release pins a version of the parser library. In that version, the rewriter visitor examines call nodes and recognizes RANDOM without regard to letter case. It also stops traversal beneath an ordering term. That implements the documented exception for ORDER BY RANDOM().

The parser's call serialization code can evaluate a call before producing its string form. Its supported random evaluation obtains an integer value and writes a numeric representation. Unsupported function evaluation falls back to normal function-call serialization. The transformation therefore retains ordinary calls while replacing the supported random expression.

This gives a precise account of what changes in the SQL. The output is a rewritten statement, not an instruction asking each replica to seed its own generator identically. Once serialized, the selected integer is ordinary SQL text that each local engine can evaluate consistently.

The transformation also means that evaluation happens per parsed call occurrence. It does not defer that occurrence to each row visited by SQLite. An author expecting a new random value for every row of a multi-row operation needs to account for the fact that the rewritten expression is now one literal at that position in the statement.

Ordering expressions retain their original behavior

The documented exception for random ordering is visible in both implementation and tests. The rewrite tests expect a SELECT ordered by RANDOM() to retain that call. Replacing its ordering expression with one constant would remove the random ordering behavior the query requested.

This exception makes the supported scope important. A randomized read order is different from assigning a random stored value. The release handles the latter through literal replacement while preserving the ordering expression. It should not be interpreted as a blanket guarantee that every statement containing randomness becomes a deterministic program.

The tests also check a plain SELECT RANDOM() transformation and an insert containing a random call. They use a regular expression for a signed numeric literal rather than require a specific generated value. That verifies the form of the rewritten statement without making the test depend on one random outcome.

A separate test disables rewriting and verifies that the original SQL remains unchanged. That covers the control path as well as the transformation. The feature is therefore represented as an explicit option passed to the rewriting layer.

The HTTP write paths invoke rewriting before execution

The HTTP service invokes the command rewriter on ordinary execute requests before building the execute operation. Queued writes also pass through rewriting before entering the batching flow. The receiving node can therefore select the random value before another node performs the replicated execution.

The documentation provides norwrandom as a request-time option to disable this behavior. Choosing that option leaves responsibility for the original statement's nondeterminism with the caller. It is a control over rewriting, not a way to make independently evaluated random results agree.

For writes intended to store the same data on every node, the normal rewritten path addresses the direct random-call problem. The resulting statement remains readable as ordinary SQL, which helps explain both its replay behavior and its departure from evaluating the function independently in every engine.

An ordinary local read has no replicated write to reproduce. The documentation consequently distinguishes execute requests from reads that remain local. This difference explains why the replication problem cannot be diagnosed merely by searching every query for a random call. The relevant question is whether independently evaluating that call changes state or an operation that the replication protocol expects nodes to reproduce consistently.

Other nondeterministic functions remain a separate problem

The release documentation explicitly says date and time functions using current time are not yet rewritten. It gives datetime('now') as an example of unsupported nondeterministic behavior, while functions using absolute values do not create that same problem. Supporting RANDOM() therefore does not remove the need to distinguish reproducible inputs from environment-dependent evaluation.

This boundary follows from the implementation. A parser can recognize many calls while its evaluator supports only selected transformations. Extending support requires defining when the value is chosen and how the resulting literal preserves the intended operation. It cannot be achieved merely by labeling a call nondeterministic.

The release's changelog describes this as the first support for nondeterministic functions through rewriting. That wording fits the mechanism. It adds one supported case to a statement-replication system, with explicit exclusions and a visible transformation.

I consider choosing the value before replication the useful design decision. The replicated statement carries the result of the random choice, allowing later execution and replay to use the same input. Application authors still need to understand the scope of literal replacement, particularly when one SQL expression would otherwise be evaluated repeatedly across rows.