July 3, 2023·4 min read·engineeringdragonflyclusterportability

Dragonfly exposes cluster key-slot calculations

Dragonfly 1.5 adds CLUSTER KEYSLOT and s390x support. Slot inspection clarifies client routing, while the architecture port shows where optimized internals need platform-specific implementations.

A key name can imply a placement constraint before the client sends a data command. Cluster-aware software needs to calculate that constraint and compare it with the server's view. Dragonfly's July 3 release adds CLUSTER KEYSLOT, making its key-to-slot calculation accessible through the familiar Redis command.

The release also adds s390x architecture support. These are different compatibility changes. One concerns the interaction between client software and the server. The other concerns the implementation below that protocol. Together they show how a replacement datastore has to accommodate both existing management tools and the hardware executing its optimized code.

I prefer Dragonfly where using more of one machine simplifies a cache deployment. A cluster-compatible interface can still be relevant in that arrangement because some clients expect cluster commands. Slot inspection is useful even when the deployment does not distribute data across several primary servers.

A slot is distinct from a server address

The key-slot patch adds a handler that accepts one key, calls the existing ClusterConfig::KeySlot calculation, and returns an integer. It does not move the key, change ownership, or return a network endpoint. That makes the operation useful for understanding placement without changing the dataset.

Its tests use somekey and expect slot 11058. They also verify that prefix{somekey}suffix maps to the same slot. A further assertion compares two differently prefixed and suffixed keys containing the same braced substring. The examples make hash-tag behaviour observable through the command. The corresponding inspection commands are:

redis-cli CLUSTER KEYSLOT somekey
redis-cli CLUSTER KEYSLOT 'prefix{somekey}suffix'
redis-cli CLUSTER KEYSLOT 'abc{def}ghi'
redis-cli CLUSTER KEYSLOT '123{def}456'

The first pair has an expected result supplied by the patch's test. The second pair demonstrates equality without requiring the caller to know its numeric result in advance. This is a practical way to inspect whether related key names express the intended placement relationship.

Sharing a hash tag also creates a concentration of keys in one slot. That relationship can be desirable for operations that require related keys to share placement. It is still a naming decision that affects distribution. The command reveals that relationship, but it does not decide whether the resulting concentration suits the application's data model. Comparing the two slot results is an observation about the naming rule. It is not an observation about traffic volume, key size, or the resources consumed by the keys that share that rule.

Emulated mode retains the slot interface

The cluster implementation at the release tag accepts emulated and yes as nonempty cluster-mode settings. In emulated mode, its topology information maps the complete slot range to one primary. Multiple slots therefore do not imply multiple independent servers. An illustrative local startup is:

./dragonfly --bind=localhost --cluster_mode=emulated

With that process running, a cluster-aware client can inspect slot and node information even though the primary owns the whole range. The emulated topology can include replicas reported by the server's replication machinery. It preserves a client-facing topology vocabulary without requiring separate primary owners for different portions of the range.

The command dispatcher checks that clustering is enabled or emulated before reaching KEYSLOT. An ordinary standalone process therefore does not provide this command merely because hashing a key is computationally possible. The cluster-mode gate is part of the observable command behaviour in this version.

The source separately implements slot information, node information, and key-slot calculation. These answer different questions. A slot number identifies the result of hashing, while topology replies associate ranges with configured nodes. A diagnostic tool needs both pieces to explain where a request should go.

Management uses a separate command path

Dragonfly's cluster-management commands use the DFLYCLUSTER prefix. The implementation restricts that path to an administrative connection. It distinguishes inspection commands exposed through CLUSTER from operations that change the Dragonfly configuration or flush owned data.

The configuration path parses JSON and constructs a new configuration object. A mutex prevents concurrent configuration updates, and the new shared configuration is installed across the process's worker threads. Slot ownership is therefore shared state that needs coordinated replacement even though dictionary data has internal ownership.

The release also includes a fix for cross-slot handling in multi-key operations. That places the new inspection command beside enforcement work. Calculating a slot is useful to clients, but the server must still apply the relevant ownership rules when executing a command.

I find that separation helpful when reading a compatibility claim. Returning a familiar topology reply and implementing the rules for actual requests are related requirements. The new key-slot command makes one calculation inspectable, while the surrounding handlers show where topology and administration remain distinct.

Architecture support reaches optimized internals

The s390x port in pull request 1214 changes more than a build-target name. It adds a vector implementation of dictionary fingerprint comparison and an architecture-specific ASCII-validation path. These operations participate in the engine's optimized handling of keys and strings.

Fingerprint comparison replicates a target byte across a vector, compares it with stored fingerprint bytes, and constructs a result mask. The purpose remains the same across architectures, but the intrinsics used to express it differ. The port supplies the s390x operations instead of assuming that the existing x86-oriented helper can compile everywhere.

The ASCII-validation changes process vector-sized portions and then inspect the remaining bytes. The patch series refines the treatment of high bits and moves the vector include into the portability header. It also updates the Lua build patch for the architecture. Runtime dependencies and low-level helpers both need attention in a complete port.

Nothing in the release establishes that every binary package or container artifact now exists for this architecture. The source supports the narrower claim that the architecture port is included. That distinction matters to deployment planning because source compatibility and a particular distribution channel are separate deliverables.

Version 1.5 consequently broadens two specific forms of compatibility. Clients gain a command that exposes key-slot calculations, including hash tags. Builders gain s390x-specific implementations for optimized engine paths. The slot calculation remains independent of the number of primary nodes, and the architecture changes remain independent of how a deployment obtains its binary.