Redis 4.0.4 persists Lua scripts for replication after restart

Resuming a replication stream requires more than knowing the next byte position. If that stream invokes a Lua script by its digest, the receiving server must also know the script body. Redis 4.0.4 addresses a restart case in which the dataset and replication metadata survived, but the script cache needed to execute the continuation did not.

The 4.0 release notes date the release to 30 November and classify its urgency as critical. The 4.0.4 notes explain that it follows 4.0.3 on the same day because the earlier release omitted several fixes its changelog had described. Persisting scripts is the specific correction completing that follow-up.

I value Redis's ability to reuse a replica's existing dataset after a restart. That optimization makes the completeness of the restored execution state important. A matching dataset at startup does not prove that the server can interpret every operation it will receive next.

EVALSHA depends on a previously known body

The release notes describe a replica restarting and receiving EVALSHA for a script it does not know. A digest identifies the script, but it does not contain the body. If the primary believes the replica already received the script, the backlog can contain digest-based invocations without another copy of the original script text.

Before a restart, the replica can have both the dataset and that cached script. Saving only the dataset and the replication continuation metadata loses the latter dependency. A successful PSYNC2 handshake can then resume precisely where the replication protocol expects while leaving the scripting engine unable to apply what follows.

That is why the release notes call this a dataset-corruption risk after a successful handshake. Establishing a shared stream position is necessary, but the position alone does not establish that every command can execute with equivalent meaning. Cached executable state participates in the interpretation of the stream.

The correction therefore belongs to persistence as well as replication. The server needs to restore the script knowledge that the upstream's choice of EVALSHA assumes. Changing the handshake to report success more accurately would not itself provide the missing body.

Replication-aware snapshots retain script bodies

The initial persistence patch saves the script cache when storing replication information in an RDB file. Its comment explicitly ties that condition to processing EVALSHA commands in the backlog after a successful partial resynchronization following restart.

The patch uses an RDB auxiliary field named lua. Its commit message explains the compatibility decision. Older versions must discard auxiliary fields they do not recognize, so this representation can carry new metadata without requiring those readers to interpret it as ordinary key data.

A follow-up patch simplifies what the auxiliary field stores. The first form combined the SHA1 identifier and script body. The final form saves the body alone and reconstructs the digest during loading. Since the identifier can be derived from the body, persisting both would duplicate information.

The loader passes the body to luaCreateFunction without a supplied function name. That function computes the SHA1 and forms the internal name using the f_ prefix. The patch trades some computation during reconstruction for less information stored in the RDB file, as its commit message states.

This reconstruction is not the same as executing the script against the dataset during loading. It makes the function available to subsequent invocations. The snapshot already represents the data at its saved point; the script body supplies execution state needed for commands after that point.

I find that distinction useful because restoring cached code can sound like replaying old writes. The patch restores the scripting engine's knowledge, so the continuation stream can perform the later operations it actually contains. Dataset restoration and script registration have different roles in that sequence.

The test checks both continuation and data equality

The regression test creates the failure conditions deliberately. It first runs a script while the replica is online and follows it with a digest-based invocation. After the datasets agree, it prevents the replica from receiving updates and sends further EVALSHA calls to the primary.

Those later calls enter the backlog while the replica is disconnected. The test rewrites the replica's configuration, restarts it, and reconnects it to the proper upstream. This makes restart persistence part of the scenario, rather than merely reconnecting a process whose script cache remains in memory.

The assertions check two outcomes. First, the primary's full-synchronization count must remain unchanged, establishing that recovery did not obtain a new full dataset. The primary and replica debug digests must then match. Successful reconnection alone would not establish the second property.

Checking both is necessary for this particular bug. A full resynchronization could hide a defect in continuation by replacing the replica's state. Conversely, a successful partial handshake could leave data divergent. The test requires the intended recovery mechanism and the resulting data agreement together.

The follow-up release contains the complete set

The 4.0.4 notes say 4.0.3's changelog had described fixes absent from its commit list. They name the script-persistence commits along with supporting corrections to function creation and the restart test. The release therefore has a narrower immediate purpose than another major redesign of PSYNC2.

The same day's 4.0.3 notes describe other replication, LFU, and module corrections. Those remain context for the maintenance line, but the reason for 4.0.4 is the incomplete earlier set. The maintainers explicitly request upgrading to the follow-up because the earlier release did not contain everything its description promised.

For replication after restart, the resulting requirement is concrete. A continuation-capable snapshot must preserve the script bodies needed by digest-based invocations, in addition to the dataset and stream metadata. The regression test then verifies that a restarted replica can consume that backlog without a full synchronization and reach the same dataset as its primary.