Redis 4.0.6 makes duplicate script loading a normal operation

Persisting Lua scripts for replication recovery creates a new ordinary case for the loader. A server receiving an RDB file can already know one of the scripts that file contains. Redis 4.0.6 changes function creation so that encountering an existing script succeeds by default, instead of requiring callers to avoid or specially permit duplicates.

The 4.0 release notes date 4.0.6 to 4 December and classify the release as critical. The versioned notes say more errors remained in the preceding PSYNC2 fixes. They also identify a critical quicklist issue at very large memory scale. Both changes concern assumptions that stopped holding under an expanded use of existing code.

I value recovery paths that can safely encounter state already present. Replication and restart handling repeatedly reconstruct information, so duplicate script registration is a foreseeable input. Making the shared function handle that case directly gives every caller the same rule.

Persisted scripts exposed an old precondition

Redis 4.0.4 had added Lua script state to replication-aware RDB files so restarted replicas could execute digest-based invocations in a resumed backlog. Its implementation could crash when an RDB file supplied a script the receiving engine already contained. The 4.0.5 notes describe that defect, and 4.0.6 addresses further errors in the attempted correction.

The first corrective patch identifies both duplicate loading and a memory leak. It changes how the duplicate lookup constructs the dictionary key and delays allocating a function-definition string until after the early duplicate return. That ordering avoids allocating a temporary string on a path that immediately exits.

The dictionary-key detail matters because the script identifier has a particular string representation. A pointer into an ordinary function-name buffer is not interchangeable with the string object the dictionary expects. Constructing the appropriate SDS value makes the lookup use the representation the script dictionary actually stores.

This patch is followed by a broader refactoring in the same release. The final API removes the need for a separate duplicate-permission argument. That is the more durable change in the caller contract, because duplicate handling becomes an unconditional responsibility of the shared function.

Function creation computes and returns the identifier

The API refactoring makes luaCreateFunction derive the SHA1 from the body every time. It looks up that identifier before compiling a new function. If the script is already present, it returns the existing dictionary key as a successful result.

For a new script, the function compiles and registers the body, retains the body through its reference count, and returns the stored identifier. On failure it returns null and, when a client context exists, reports the error to that client. Callers no longer supply a separately constructed internal function name.

The returned identifier remains valid until the scripting subsystem resets, according to the function's comment. That lifetime is part of the API contract. A caller can use the result to reply or continue its operation without treating it as a separately allocated permanent copy.

SCRIPT LOAD now uses this same function instead of separately computing a digest and checking for an existing entry. The RDB auxiliary-field loader also calls it. Centralizing those operations removes duplicated knowledge of how to name and register a Lua function.

The commit acknowledges that first-time EVAL processing can recompute the body digest. Its reasoning is that defining a new script is an infrequent event compared with repeated execution and that Lua compilation already has a cost. This is the maintainer's rationale for the API simplification, without requiring an unsupported throughput claim.

I find the shared duplicate rule preferable to adding more caller-specific exceptions. SCRIPT LOAD and RDB loading both legitimately encounter existing scripts. The final function treats that condition as success and leaves compilation to the path that actually needs a new definition.

The regression test forces a full resynchronization

The new test changes a replication identifier and kills the upstream connection. This forces a replica to receive a full resynchronization containing RDB lua auxiliary fields. The comments identify the important condition as receiving scripts already present in memory.

The test waits for the replication link to come up and then compares the datasets' debug digests. Those checks establish both that loading completes and that the resulting data agrees. A loader that merely avoids an assertion but fails to restore equivalent state would not satisfy the digest check.

The release notes say the fix received independent review by two developers and Valgrind testing. They also emphasize making duplicate tolerance the default behavior so it can be exercised more directly. Those are concrete verification steps reported for this release, rather than a claim that every possible scripting defect is resolved.

Quicklist counts need consistent widths

The other critical correction concerns large lists. One quicklist patch changes quicklistCount to return unsigned long instead of unsigned int. The structure already stores its total entry count as unsigned long, so the function's old return type could narrow that count.

A second patch widens the quicklist's node-count field from unsigned int to unsigned long. It updates the diagnostic formatting to match. The total number of entries and the number of quicklist nodes are different counts, and the release corrects their widths in separate places.

These patches explain the notes' reference to massive memory scale without suggesting that ordinary small lists trigger the same boundary. The exact size of C integer types depends on the platform, so the type changes alone do not justify a universal dataset-size threshold.

Redis 4.0.6 therefore corrects both reconstruction and representation assumptions. Script loading accepts an already known body through one shared API. Quicklist accessors and fields retain the wider counts their stored structures require. The release recommends upgrading any 4.0.x installation, with particular emphasis on 4.0.4 and 4.0.5 because of the scripting failures introduced by their replication fixes.