Dragonfly expands Redis snapshot loading

A server can understand the commands used by an application and still reject the file that contains its existing data. Redis snapshot compatibility depends on serialized representations as well as on logical data types. Dragonfly's August 2 release announces support for RDB files produced by Redis 6 and Redis 7.

The implementation makes that announcement more specific. It raises the recognized RDB format version and adds handling for newer compact encodings. It also resolves a conflict between a Dragonfly-specific type number and a newer Redis type. These are file-format changes below the application command interface, and they determine whether an existing dataset can enter the server through a snapshot.

I find this work relevant to Dragonfly's argument for simpler scaling. Replacing a cache engine is easier when its existing data has a direct import path. The path still needs a defined compatibility boundary, particularly when the source file contains more than ordinary string keys.

A file version is only the first check

The release-tagged loader begins by reading a nine-byte header. It checks the five-byte Redis signature and parses the four-byte version field. Versions below five or above the implementation's supported maximum produce a version error.

The format-support patch raises RDB_VERSION from nine to eleven. This changes the header-level gate, but accepting a version does not establish that every opcode within such a file has an implemented decoder. The load loop and object dispatcher still decide what the server can reconstruct.

That separation matters when interpreting compatibility. A file identifies its format version once, while individual objects can use different encodings. Two snapshots with the same header can consequently exercise different loader branches. A dataset containing only strings does not demonstrate that a snapshot containing every supported Redis data type will follow the same path.

The loader also handles attributes that precede an object. Millisecond expiration timestamps become settings associated with the next value. Database-selection opcodes switch the destination database, and the implementation flushes pending shard work before that transition. Importing values therefore includes reconstructing their database context and expiration metadata. The database switch also shows that parsing and installation are separate activities. Pending objects can already be queued for workers while the parser encounters the next control opcode, so the loader coordinates those activities before changing context.

Some metadata is deliberately ignored. The source consumes frequency and idle-time information without treating it as application data. That choice illustrates another compatibility distinction. Preserving values and expiration times does not require the receiving engine to reproduce the source engine's internal eviction history.

Compact containers need their own decoders

Redis can serialize a logical collection using a compact representation. The format-support work adds paths for hash listpacks and set listpacks. Those object tags require the loader to interpret the compact container, recover its elements, and place them in the receiving engine's representation.

The tagged object dispatcher handles old hash ziplists and hash listpacks through a generic serialized-object reader. The construction path then interprets the container according to its type. Support for a logical hash therefore includes several possible serialized layouts, not a single universal sequence of field-value strings.

For older ziplist hashes, the loader validates the structure while converting entries to a listpack. Its callback checks field-name duplication and requires an even number of entries for field-value pairs. These checks are part of reconstructing a meaningful hash from bytes, rather than assuming that a valid header makes every later object valid.

A compact set has a different uniqueness rule from a hash's alternating fields and values. The implementation converts set elements into the chosen set representation. The serialized form and the runtime representation can therefore differ without changing the set operations an application sees after loading.

This is why adding an encoding constant to a header is insufficient. The loader has to dispatch the tag, decode the bytes, and construct a value whose runtime behaviour matches the logical collection. Each stage has a separate opportunity to reject malformed or unsupported data.

A type-number collision needs version context

Dragonfly previously used type number twenty for its JSON representation. The newer Redis format uses that number for a listpack-encoded set. The patch moves the Dragonfly JSON type to thirty and retains an old JSON type identifier for earlier files.

The tagged loader uses the RDB version to resolve the overlap. For versions below ten, the old identifier can mean Dragonfly JSON. With newer format versions, that same numeric tag represents the newer set encoding. The file's header therefore influences the meaning of an object tag later in the stream.

This is a concrete example of versioning doing semantic work. A decoder cannot interpret type twenty correctly by looking only at the next object byte. It needs context established when the file opened. Moving the custom tag avoids the continuing collision while the conditional preserves the older interpretation where required.

The distinction also separates Dragonfly JSON from arbitrary Redis module serialization. The main load loop rejects module auxiliary data with a feature-not-supported error. Having a built-in JSON representation does not mean that the loader can reconstruct every module's private format.

I regard that explicit rejection as preferable to an ambiguous import claim. The useful compatibility statement names the encodings the loader understands. It does not imply that the receiving engine has acquired the runtime extensions or private object models of the source server.

Signed integers are part of data fidelity

The release includes a separate small-integer loading fix. It changes sixteen-bit and thirty-two-bit reads from unsigned to signed types. A serialized negative integer has to retain its sign when the loader converts it back into the value used by commands.

The patch also removes a duplicated integer-decoding path. A helper that reconstructs an integer as a string now calls the common integer reader. That prevents two implementations of the same encoding from diverging in how they interpret the sign bit.

The regression test stores negative values in hash fields, reloads the database, and reads them back. Its examples include negative five, negative five hundred, and negative fifty thousand. These values exercise different integer widths instead of testing only a small positive value that both signed and unsigned decoding would preserve.

A minimal application-level round-trip example for the values is:

redis-cli HSET import-check small -5 medium -500 large -50000
redis-cli HGET import-check small
redis-cli HGET import-check medium
redis-cli HGET import-check large

The source's regression test adds the reload between writing and checking. The important property is equality across serialization, not merely success of the original writes. A snapshot loader can complete without a format error and still be wrong if it reconstructs a different numeric value.

New format support still has explicit limits

The tagged type dispatcher supports the original stream-listpack encoding, but it does not contain cases for the newer stream-listpack variants defined in the updated format header. Those tags reach its unsupported-type branch. A higher accepted RDB version therefore must not be read as support for every stream encoding associated with that version.

The same source rejects an older seconds-resolution expiration opcode, while it implements the millisecond form. This is another reason to inspect the file's actual encodings when explaining an import failure. A broadly recognizable file can contain an unsupported construct before the loader reaches ordinary objects.

The header and dispatch distinction is especially relevant to migration tools. A preflight that checks only the leading signature and version can confirm that the file is a candidate. It cannot prove that all subsequent objects have receiving-side decoders. The object-level paths remain the authority for that question.

Snapshot import is also a point-in-time transfer. Reading the file reconstructs the state represented in that file. The loader's ability to parse it does not by itself establish how writes made on the source after the snapshot will reach the destination. That is a separate replication or cutover question.

The release broadens an existing interface

The same release announces partial geospatial command support and TLS improvements. The primary changes for snapshot migration are narrower and more concrete. New compact-container paths broaden the data that can be reconstructed, while the JSON tag change preserves context across file versions.

Version 1.7 makes the snapshot boundary easier to cross for ordinary Redis datasets. The loader's signature check, version gate, and object dispatch explain the stages involved. Signed integer decoding supplies a separate correctness requirement within an accepted encoding, and the unsupported stream and module paths define where this release's import support ends.