January 22, 2024·5 min read·engineeringdragonflyaclcaching

Dragonfly adds key permissions to its Redis compatibility

Dragonfly 1.14 adds key-pattern permissions and the first client-tracking implementation. The changes show how much Redis compatibility depends on behaviour around each command.

Allowing a connection to execute GET says nothing about which application's keys that connection should read. That distinction becomes important when several applications share a database process. Dragonfly's key-permission work adds a second authorization decision alongside permission to run a command. A user can have string-command permissions while still receiving an error for a key outside its permitted patterns.

The Dragonfly 1.14.0 release, published on January 22, includes that work alongside initial client tracking and tighter memory accounting. These changes concern the behaviour surrounding command execution. For an application adopting a compatible database engine, that behaviour can determine whether an existing security policy or client library still works.

I find Dragonfly attractive where scaling a cache within one machine can simplify operations. Its usefulness in that role also depends on preserving the application assumptions attached to the Redis protocol. Key permissions are a concrete example of an assumption that successful command parsing alone cannot satisfy.

Command permissions and key permissions are separate

The ACL key-support issue describes glob-based permissions, a wildcard for all keys, and a way to reset existing key rules. The implementation tests make the separation between commands and keys explicit. An account with access to administrative commands and all keys still cannot execute SET without the relevant command permission. Conversely, granting string commands does not authorize arbitrary keys when the key patterns remain restricted.

That distinction changes the meaning of a shared process. An application may need ordinary string reads and writes without needing access to another application's namespace. A prefix convention alone cannot enforce that boundary. Once authorization checks the supplied keys, the convention becomes something the server can reject a command for violating. This still requires a deliberate naming scheme, because the policy matches key names.

The validation tests include literal keys and glob patterns such as ~bar*. They also check separate read and write permissions. The following commands illustrate the same split using the syntax exercised in those tests. They are protocol commands for an administrator to send, with the example credential replaced before actual use.

ACL SETUSER report-reader ON >replace-this-password resetkeys +@string %R~report:*
ACL SETUSER report-writer ON >replace-this-password resetkeys +@string %W~report:*

The command category supplies permission to execute string operations. The key rule then constrains which of those operations can access matching keys. In the tests, a read-only key rejects a write, and a write-only key rejects a read. This is more precise than assigning the same unrestricted namespace to every account that needs a common command family.

Multi-key commands require particular care. The validation tests include a ZUNIONSTORE request whose input keys match an allowed pattern but whose destination does not. Dragonfly rejects the command. Authorization therefore has to include the output key, not merely the first key or the keys used as inputs. Application code that constructs destination names independently of its input names can encounter a permission error even when the inputs look correct.

The feature also has an administrative lifecycle. The release links changes for reading key rules through ACL commands, listing them, saving and loading them, and recording key-related authorization failures. Permission enforcement without a corresponding way to inspect and preserve the rules would be difficult to operate. Here, the surrounding changes make the policy visible through the existing ACL interfaces.

Client tracking connects reads to invalidations

Client-side caching requires a relationship between a value an application has read and later changes to that value. The first client-tracking pull request describes a map from keys to sets of tracking clients. Dragonfly records the keys involved in a read-only command so it can identify the clients interested in those keys.

That mapping explains why this feature belongs inside command execution. A client library cannot obtain authoritative invalidations merely by remembering its own reads. The server must connect subsequent mutations to the clients that hold copies. For multi-key reads, tracking must cover the keys the command actually references. The first implementation also adds a transaction-reuse interface for that work.

The second tracking pull request supplies the invalidation notifications and functional tests. Recording readers and notifying them are separate implementation steps, but an application needs both for a useful coherence protocol. The release describes the result as initial support. That wording is a reason to avoid treating this version as a claim that every client-tracking mode or client-library expectation has been implemented.

I regard server-assisted invalidation as a useful extension of a cache engine because it lets the application retain local copies without relying solely on a chosen lifetime. It also introduces server state proportional to the tracking relationships. The release includes a separate change to account for memory used by the tracking table, which makes that cost relevant to the memory-accounting work in the same version.

Accounting includes the connections around the data

The connection-memory correction adds missing fields and fixes initialization of replication information so that replication connections enter the accounting correctly. This is a narrower and more useful claim than saying the release reduces memory consumption. Correcting a counter can raise the reported usage without changing the workload's allocation behaviour.

That distinction matters when comparing measurements across an upgrade. A higher figure can reflect memory that the previous accounting omitted. Client tracking creates another reason to inspect what the counters include, since stored values are only part of the engine's responsibilities. Connections and bookkeeping also consume capacity, and the release's fixes bring more of those allocations into its measurements.

The stream change has a defined limit

The release also advertises a blocking stream-read correction, but the underlying pull request is more qualified. It says it partly fixes the referenced issue and corrects the requested range for blocking operations. Its description discusses the consumer-group > identifier and leaves some expected test results pending a blocking-mechanism fix.

That source does not support a claim that blocking stream compatibility is complete in this release. It supports a correction to one part of the read path. The distinction is useful for applications that depend on blocking behaviour, because choosing the correct range and waking a blocked reader correctly are different requirements. Version 1.14 advances those surrounding compatibility obligations while leaving the stream issue explicitly partial.