Redis 4.0 preserves replication history across failover

A replica can contain almost all the data it needs and still require a full resynchronization if the new upstream cannot establish their shared history. Redis 4.0 changes that recognition with PSYNC2. It retains information about an earlier replication identity so a promoted replica can accept some continuation requests that would previously have fallen back to full synchronization.

The 4.0 release notes date the first stable release to 14 July. The notes at the 4.0.0 tag describe substantial internal changes, including replication and threaded deletion. Loadable modules also become part of the stable major release. The operational story includes how nodes recover and extend their behavior, alongside new commands.

I like Redis's data model, but continuity during a topology change is part of its practical cost. Reusing an existing dataset is useful only when the server can justify continuing the same replication history. PSYNC2 makes that justification more flexible while keeping explicit limits on which history and bytes are available.

A replication identity describes history

The 4.0.0 replication implementation checks two things before accepting a partial resynchronization. A requested replication identity must match an acceptable history. The requested offset must also fall inside the retained replication backlog. Knowing the identity without retaining the needed bytes is insufficient.

The new mechanism permits two potentially valid identities. The primary identity describes the current history. A secondary identity remains valid only up to a recorded offset. That limit prevents a server from treating every future position in an older history as part of its current dataset.

masterTryPartialResynchronization first compares the requested identifier with those identities and checks the secondary cutoff where necessary. It then checks that a backlog exists and covers the requested position. Failure at either stage takes the full-resynchronization path. Partial recovery remains conditional, rather than becoming a promise for every reconnect.

When the checks succeed, the server marks the client as an online replica, returns CONTINUE, and sends backlog data from the requested position. A PSYNC2-capable client also receives the current replication identity in that response. This lets continuation convey a history change as well as the missing bytes.

Promotion keeps a bounded earlier identity

The shiftReplicationId function explains the failover case. When an instance changes from replica to primary, it copies its current replication identity into the secondary slot and creates a new current identity. Other replicas that know the former primary's identity can then request continuation against the retained shared portion.

The secondary limit is the current offset plus one. The source comment explains why the addition matters. A replica asks for the first byte it has not received. If two instances agree through byte 50, a request beginning at byte 51 is consistent with that shared prefix even though the stored position of completed history is 50.

This is a byte-stream boundary, not a count of keys or commands. A new primary must distinguish the history it inherited from the history it generates after promotion. Keeping the earlier identity with an offset limit expresses that distinction without pretending the old primary and the new primary can independently generate the same future stream.

The implementation also invalidates the secondary identity when a new history begins after a full resynchronization. Creating a backlog from scratch uses a new identity and clears the secondary one because the server lacks a valid retained past. The bookkeeping follows the available evidence of shared history, not merely the names of the participating processes.

I find this the most useful detail in PSYNC2. A role change does not automatically destroy every opportunity to reuse data. The implementation records which earlier history remains meaningful and where that claim stops, then separately requires the backlog bytes needed to continue.

Chained replicas follow the upstream stream

The release notes also describe a change to chained replication. Each downstream replica receives the same stream generated by the top-level primary. The notes associate the new design with partial resynchronization after failover and after a replica restart, including cases where a former primary becomes a replica of one of its former replicas.

Those cases share a requirement to recognize continuity despite a change in process role or connection. The primary and secondary identities describe the stream's history, while the offsets locate the continuation point. The release does not remove the need for full synchronization when the relevant history or backlog is unavailable.

The notes describe backward compatibility for a 4.0 instance acting as a replica of an older server. That is a replication-protocol statement. It should not be generalized to every protocol used by Redis Cluster, because the same release documents an incompatible change to the cluster bus.

This distinction matters when planning the meaning of an upgrade. An application can keep using familiar commands while node-to-node compatibility changes. Client command compatibility, replication behavior, and cluster-bus compatibility need separate explanations because they involve different exchanges.

Modules add server-owned commands and types

Redis 4.0 also provides the module system described during the release candidates. The contemporary module introduction explains dynamic libraries loaded into the server and initialized through an API version. Modules can register commands and implement data types through the C interface.

This moves extension work beyond a client convention for arranging ordinary keys. The module owns operations and potentially a representation inside the server. A Redis client can invoke the resulting commands, while deployment must provide the corresponding native library and initialization.

The stable release notes add thread-safe module contexts. They call the API experimental while considering it stable and usable when needed. That wording describes the interface available at this release; it does not make every module command execute concurrently or make arbitrary access to server state thread-safe.

The native module data-type RDB format also changes relative to the preceding candidate. That is relevant for candidate users carrying module data forward. The stable server release and the persistence representation used by a module need to agree, so an extension interface has storage consequences as well as command names.

Cluster upgrades have a separate protocol boundary

The stable release announcement explicitly says that the Redis 4.0 cluster bus cannot communicate with the 3.2 bus protocol. It requires a mass restart of cluster instances and rules out a rolling upgrade across that boundary. The notes explain the reason as support for containers and NAT, where assuming a bus port at a fixed offset was inadequate.

This is a material operational exception to the broad compatibility of exported commands. An application may need little adjustment while its cluster requires a coordinated restart. PSYNC2's improved continuation does not make the incompatible cluster-bus versions interchangeable during that transition.

The migration section also changes the address format in CLUSTER NODES to host:port@bus-port. Clients using that output must ignore the bus-port suffix as documented. The notes recommend CLUSTER SLOTS for obtaining cluster configuration. A parser that assumes the old address spelling can therefore fail even if the underlying application commands remain supported.

Other interfaces have smaller changes. HSET becomes variadic, and the notes recommend it for new code while retaining HMSET. SLOWLOG gains client name and address fields, so consumers should not assume the previous entry length. These are specific examples of why compatibility needs to be assessed per interface.

Reclamation can happen outside the immediate delete

The release retains asynchronous deletion through UNLINK and the ASYNC forms of database flushing. The major feature notes explain that Redis can free keys in a background thread instead of performing all reclamation on the command path. This addresses expensive deletion work without changing the replication-history checks described above.

Mixed RDB-AOF rewriting is another separate mechanism. When enabled, it writes an RDB representation first and appends an AOF stream, with the stated aim of faster rewrites and reloads. The AOF checking utility gains support for those RDB preambles in the stable release.

Redis 4.0 therefore changes both the extension model and several maintenance operations. For replication, the central improvement is the bounded recognition of an earlier history after role changes. A continuation still needs a valid identity, an acceptable offset, and retained backlog data. The cluster upgrade rules remain an independent constraint because the bus protocol changes even where application commands continue to work.