September 8, 2022·4 min read·engineeringdragonflysnapshotsjson

Dragonfly 0.7 schedules snapshots and adds JSON operations

The September release adds UTC snapshot scheduling and parses JSON from stored strings, while Lua modules broaden script compatibility.

Automatic snapshot timing and structured-value access address different gaps in a datastore, but both need precise semantics. Dragonfly 0.7.0, released on September 8, adds scheduled snapshots and preliminary JSON support. Its examples let an operator schedule existing persistence work and let a client query JSON contained in a string value.

The release also adds Lua modules and improves memory-limit checks. Those additions matter because compatibility includes the environment around a command as well as its name. A script can use supported database operations and still fail if its expected serialization library is absent.

I am interested in changes that make a single Dragonfly instance easier to operate. Scheduling persistence inside the server removes one external scheduling dependency, while keeping the snapshot mechanism visible. The useful question is exactly which work the server now schedules and which time basis it uses.

The schedule uses UTC

The server source at v0.7.0 declares save_schedule as a glob-style specification for UTC time in a 24-hour hour-and-minute format. An empty setting leaves the scheduler unconfigured. A nonempty value is parsed before the snapshot fiber starts, and an invalid specification generates a warning.

A fixed daily time can be expressed as an argument to the server. This example requests the documented hour-and-minute form without relying on a host's local timezone.

dragonfly --save_schedule=02:00

The relevant time is 02:00 UTC. An operator reading the final flag definition should not substitute local wall time merely because the host has a local timezone configured. The distinction is particularly important for an in-process scheduler deployed across machines with different timezone settings.

The snapshot-scheduling change contains the implementation history and tests. Early revisions included a separate timezone selection, but the tagged source defines the behavior shipped in this release. Reading only the first patch revision would therefore produce the wrong configuration advice.

The scheduler runs in a server fiber and waits between checks. The tagged implementation uses a twenty-second wait interval. That arrangement describes periodic checking of a time specification, not a promise that a snapshot begins at an exact subsecond instant.

The feature schedules snapshots. It does not by itself establish replication or failover, and the release does not present it as either. A periodic snapshot and a continuously maintained second server answer different operational requirements. The new flag gives the existing persistence operation a recurring time policy.

JSON begins with the stored representation

The release notes demonstrate storing a JSON document through SET and querying one of its members through JSON.GET. That example is informative because the new commands work with the JSON text already stored as a string.

SET foo "{\"pi\": 3.14}"
JSON.GET foo $.pi

The example follows the release's documented operation, with quoting appropriate for an interactive Redis command session. It establishes a small structured query without claiming support for an entire external module's command set.

The tagged JSON implementation first looks up a string object in the owning shard. It obtains the string data, including through the tiered-storage path when the value is external, and then parses that data into a JSON representation.

Parsing can fail. The implementation checks decoder validity and returns a syntax error when the stored value is not valid JSON. This is different from treating every string in the database as a document merely because a JSON command exists.

The query path compiles and evaluates JSONPath expressions. For one supplied expression it returns that evaluation's serialized result. For several expressions, the implementation constructs an object containing the separate query results. Those branches matter to the shape a client receives, even when the input document stays identical.

The code also has helpers for storing updated strings and recording journal entries when a journal is present. Dragonfly is therefore integrating structured operations with its existing storage representation. The release's word preliminary should remain attached to this work because a few implemented operations do not establish complete compatibility with every JSON workload.

Lua compatibility includes its libraries

Pull request 273 loads cjson, struct, cmsgpack, and bit modules into Dragonfly's Lua interpreter. The release notes describe these capabilities in broader terms, while the implementation supplies the names scripts actually use.

This matters for code that serializes data or performs bit operations inside a script. Supporting Lua syntax alone does not provide every library an existing script expects. Loading the modules supplies another part of that execution environment.

The patch includes interpreter tests for module behavior, including JSON encoding. That is a more concrete compatibility claim than saying that scripts are generally supported. A script still has to stay within the database commands and interpreter behavior the server implements.

I prefer that level of specificity because these capabilities have independent compatibility requirements. Document queries concern the stored JSON and the shape of the result. Lua modules supply functions that a script can call, while scheduling establishes when persistence starts. Progress in one does not establish completeness in another.

Memory limits also receive attention

The September notes say the release closes several paths that let memory grow beyond the configured maxmemory limit. They also add STICK so selected items can remain in the store when cache mode is enabled. These changes concern admission and eviction behavior alongside the new persistence and query facilities.

The release also includes a local monitoring setup under its tools directory. The notes describe a Docker Compose configuration for Grafana and Prometheus that observes a local Dragonfly instance. It supplies an example deployment for inspection rather than changing the semantics of the snapshot schedule or JSON query.

The combination makes the release more operationally relevant than a list of new commands suggests. A server can schedule its snapshots, expose structured operations over stored JSON strings, and provide more of the Lua environment existing scripts expect. For scheduled persistence specifically, the final UTC flag definition gives operators the time semantics needed to configure the feature correctly.