Redis 4.0 RC3 tracks expiration on writable replicas

A replica normally follows the upstream server's decisions about removing expired data. Allowing local writes complicates that rule because the replica can create expiring keys the upstream does not know about. Redis 4.0 RC3 addresses leaked keys in that configuration by tracking local expiration separately. The fix makes the exception explicit in the implementation.

The 4.0 release notes date RC3 to 22 April and give it the internal version 3.9.103. The notes at the RC3 tag identify writable-replica expiration and a MIGRATE TTL defect among the corrections. They classify the update as high urgency for users of RC2, while retaining the release-candidate status.

I like Redis's explicit lifetime operations, but the identity of the server responsible for expiration matters. A writable replica has two sources of data and cannot apply the ordinary upstream-driven rule indiscriminately. The implementation's limitations help explain what that configuration actually supports.

Local expiration has its own dictionary

The tagged expiration source introduces a dictionary named slaveKeysWithExpire. Its keys are copied key names. Its values are 64-bit bitmaps describing the databases in which a locally expiring key may exist. A key name can therefore be tracked in more than one database without creating a separate dictionary entry for every database.

rememberSlaveKeyWithExpire initializes that dictionary on first use and sets the bit corresponding to the database. The comments identify it as tracking keys that receive EXPIRE or similar commands in the context of a writable replica. This is bookkeeping for local expiration, separate from a general scan of every replicated key.

The copied key name has a purpose. It lets the tracking dictionary retain its own identifier without requiring the entry to remain synchronized with the main database's key allocation. The expiration scan later removes tracking entries when they are no longer needed. That gives the bookkeeping a lifetime independent of the value it describes.

The representation also has an explicit limit in this candidate. rememberSlaveKeyWithExpire returns immediately for database identifiers above 63. The source comment acknowledges that keys created in those databases are not expired by this mechanism. A 64-bit bitmap is a compact representation, but the code does not silently extend it to an arbitrary number of databases.

The expiration scan follows those local records

expireSlaveKeys chooses a tracked key and examines the databases marked in its bitmap. For each relevant database, it looks up the expiration entry and calls the active-expiration helper when appropriate. Entries that still have an unexpired value retain their database bit. If no bits remain, the key leaves the tracking dictionary.

This design lets one key name disappear from one database's tracking while remaining relevant in another. The bitmap after the scan describes the remaining expiration work. It is not merely a flag saying that a key with that name existed at some earlier point.

The loop also has bounded stopping conditions. It can stop after encountering enough unexpired entries, when its periodic elapsed-time check exceeds the small time budget, or when the tracking dictionary becomes empty. This keeps the maintenance work incremental instead of requiring every local expiration to be examined in one pass.

The active-expiration helper removes an expired key, emits the expiration notification, and increments the expired-key statistic. It selects synchronous or asynchronous deletion according to the configured lazy-expiration setting. Tracking which keys need local expiration and deciding how to reclaim their memory are separate parts of the path.

Local writes still have consistency limits

The source clears this tracking dictionary when data is flushed. Its comment explains that new replicated keys may arrive with the same names and should not inherit the old local-expiration bookkeeping. Keeping an old record could otherwise attach an earlier local interpretation to a later upstream value.

The same comment calls the behavior best effort and warns that using the same key names for writes on both the replica and its upstream can produce inconsistencies. This fix does not turn a writable replica into a general conflict-resolving multi-writer database. It addresses the lifetime of locally expiring keys under the documented model.

I find that boundary useful when reading the release headline. Fixing leaked expiration is a specific improvement, while the source still identifies races associated with overlapping local and upstream names. Those two facts can hold at the same time without diminishing the value of the correction.

The release also fixes migration failing to preserve a key's TTL under certain conditions. Expiration metadata is part of a key's behavior, so moving a value without its lifetime can change the application's dataset. The notes identify the correction without claiming that every expiration issue has the same cause as the writable-replica leak.

Memory defragmentation enters the candidate

RC3 also adds active memory defragmentation, credited to Oran Agra. The shipped configuration describes copying values into better allocation locations and releasing the old copies incrementally. It uses jemalloc-specific functionality to identify fragmentation and improve allocation placement while the server continues running.

The configuration labels this experimental, disables it by default, and requires the bundled jemalloc. It documents enabling it at runtime with:

CONFIG SET activedefrag yes

The configuration also exposes minimum and maximum effort controls expressed as CPU percentages. Fragmentation thresholds determine when that work begins and when it uses maximum effort.

This option addresses fragmentation between allocations. It has a different role from expiring keys or moving deletion work to a background thread. A server can have memory occupied by live values, detached values awaiting release, or space that allocation history has fragmented. Those situations need different mechanisms, even though each appears in memory usage.

The new candidate therefore changes several parts of lifetime management without making them interchangeable. Writable-replica tracking identifies locally expiring keys. Migration preserves lifetime metadata during movement. Active defragmentation relocates live allocations under its documented allocator requirements. The expiration implementation's database limit and key-name overlap warning remain relevant to the precise behavior available in RC3.