Dragonfly 1.31 resumes replication from retained journal entries

A disconnected replica usually already has most of the primary's data. Replacing that data with a complete snapshot can be unnecessary if the primary still retains the changes the replica missed. Dragonfly 1.31, released on June 19, introduces partial synchronization for this short-disconnection case. Its main operational benefit is avoiding a full replica reload when the required history remains available.

The implementation restoring the path checks the replica's requested log sequence number against retained journal entries. It selects partial synchronization when that position is still available, or when the replica is already at the next position. If the requested history has become stale, the server selects full synchronization instead. A brief elapsed time alone is therefore not the condition which guarantees success.

I like Dragonfly where a larger single process can postpone a cluster, but recovery work remains part of that deployment's cost. Reloading a large replica consumes resources even when the original disruption was small. Reusing retained changes is a more proportionate recovery path when the implementation can establish continuity.

A sequence number identifies the missing suffix

The released incremental snapshot code interprets the replica's sequence number as the next entry it wants to receive. It serializes retained journal records while that position remains in the buffer, advancing the number after each entry. The requested position describes missing history rather than a request to read the current value of every key.

This distinction explains what partial synchronization saves. A full snapshot describes the dataset at a suitable point and then accounts for subsequent changes. Partial synchronization can begin with the known prefix already present on the replica and transfer only the missing suffix. The primary needs enough retained history to establish that the suffix is continuous.

The mechanism is not a general merge of independently modified databases. It relies on the replication relationship and sequence information represented by the protocol. Two datasets with similar contents are insufficient evidence that one can safely receive the other's next journal record. The server's continuity checks determine which recovery path is valid.

The implementation also has handling related to replicas which share a previous primary. That makes identity information relevant alongside numerical positions. A position in a log has meaning in relation to the history which produced it, so treating every identical number as the same replication state would be incorrect.

Retention is measured in journal entries

The journal implementation uses a circular buffer for retained records. Its shard_repl_backlog_len setting specifies the number of entries retained per shard, with a default of 1024 in this tag. The buffer tracks serialized bytes separately, because record count and memory consumption are different quantities.

That distinction matters when interpreting how long a replica can be disconnected. A fixed number of records covers more elapsed time at a low write rate than at a high one. Records can also differ in serialized size. Neither a duration nor a byte budget can be inferred from the entry-count setting alone.

The buffer tests whether the requested position lies between the first and last retained sequence numbers. Access to a particular entry depends on that retained range. As new records arrive, older ones can leave the ring. A larger dataset does not by itself determine this turnover rate, since the deciding input is the journal activity during the interruption.

An operational description should therefore connect retention to the write pattern. The feature can turn a short network interruption into replay work, but the amount of retained history limits that benefit. Calling every reconnect a partial synchronization would hide the condition which permits the server to reuse the replica's existing dataset.

Catching up and following live writes need one boundary

Finishing buffered replay is not the end of recovery. The replica must then receive future journal changes. The incremental snapshot code checks that its next sequence number equals the journal's next expected position before registering for live changes. Its comments explain the synchronization relationship which prevents processing a record twice during that transition.

If the required entry falls out of retention before the catch-up completes, the code reports a nonrecoverable partial-sync error. That is an explicit refusal to continue with a gap. A successful initial availability check cannot promise that all entries remain retained throughout an arbitrarily slow transfer.

This is a more precise account of recovery than saying the server starts reading the backlog. Replay and live subscription overlap in time with ongoing primary writes. The implementation needs to establish where one source of records ends and the other begins, with no missing or duplicate record at that boundary.

A supporting flow correction moves connection migration to the appropriate execution context before the relevant replication checks. Its test covers changing replication relationships while journal positions advance. The path depends on the shard execution context as well as the sequence values being compared.

That test is relevant to a multicore engine. A sequence lookup performed on the wrong execution context can be wrong even when the comparison expression appears correct. Ownership of per-shard journal state determines where the recovery decision must run.

Error cleanup must wait for the replay fiber

Pull request 5171 repairs finalization when an incremental snapshot encounters an error before registering a journal callback. The previous early-return condition could leave the snapshot fiber unjoined when its containing object was destroyed. The correction waits for the fiber even when the callback identifier was never set.

This separates two facts that usually occur together on the successful path. A callback may be absent because setup failed, while background execution still exists and needs to finish. Using callback presence as evidence that no work remains makes cleanup depend on an assumption the error path violates.

The repair supports the partial-sync feature's failure behavior. Refusing a replay with missing history is correct only if the rejected operation also releases its work consistently. An error result does not by itself establish that the background task has stopped using its containing state.

I find these details useful because a recovery feature needs a defined unsuccessful outcome as well as a fast successful one. The retained history can be unavailable, a connection can fail, or setup can stop partway through. The release includes changes which account for those paths in the same implementation that avoids the reload.

Other additions preserve the distinction between values and metadata

Memcached GAT and GATS support combines retrieval with updating expiration. The implementation classifies the operation as a write because it changes a key's lifetime while returning its value. A read-shaped result does not make the operation read-only when it also mutates metadata.

The command passes expiry as part of the operation's arguments, replacing an earlier approach which packed it into connection flags. This keeps the mutation input with the command performing it. The same release's partial synchronization then has to preserve such mutations, since expiration state is part of what a replica needs to reproduce.

The release also labels Huffman encoding for keys experimental. Its configuration supplies a code table for the key domain, and the compact-object implementation decodes the original string when hashing or comparing encoded keys. The stored representation changes while the logical key bytes retain their meaning to commands.

The patch's example shows a dataset whose keys fit an inline representation after encoding, removing separate key allocations for that case. That result depends on the keys and selected table. It is a concrete allocation effect, rather than a promise that all datasets lose the same amount of memory overhead.

Finally, migration out-of-memory handling adds a fatal outcome when applying incoming data would exceed the configured memory limit. The receiving node closes the migration instead of continuing to treat the situation as ordinary progress. That is separate from replica partial sync, but reflects the same need to make an unsuccessful transfer observable.

Dragonfly 1.31's recovery improvement depends on established history and a correct transition to live replication. When the requested records remain retained, the replica can reuse its existing data. When that proof fails, the implementation reports the failure or chooses full synchronization, preserving the distinction between a connection returning and replication continuity actually being established.