Redis 5 RC1 gives stream entries ordered identities

An ordered record needs an identity that survives reading it. Redis 5 RC1 introduces Streams with entry IDs and range operations, so an application can name its position in stored history. The important change is the relationship between an entry and a reader. Reading a range does not require removing the entries that another reader may still need.

The Redis 5 release notes date the first candidate to May 29. The RC1 version of those notes describes a release focused on user-facing data types and operations. It explicitly anticipates further candidates before the code is production ready. That qualification matters when discussing a data structure whose stored entries and processing state can become part of an application protocol.

I like Redis when the operation exposed to an application corresponds to a clear data model. Streams interests me because a reader can express progress using an entry identity. That is useful information to give the server directly, without requiring the application to encode every ordered-record operation itself.

An entry has fields and an increasing ID

The candidate implementation accepts field-value pairs through XADD. A caller can supply an explicit ID or ask Redis to generate one with *. The following commands illustrate the basic model on a new key:

XADD readings * sensor north temperature 18
XADD readings * sensor south temperature 21
XRANGE readings - + COUNT 10

XADD returns the identity assigned to the appended entry. XRANGE retrieves entries between its bounds, with an optional count. The example deliberately uses the candidate command syntax and does not assume any later additions to the API.

An ID has a millisecond component and a sequence component. In streamNextID, Redis uses the current millisecond time when it exceeds the last ID time, starting that new millisecond with sequence zero. Otherwise, it retains the previous time component and increments the sequence. IDs consequently continue increasing when several entries arrive within one millisecond or the observed clock has not advanced.

This mechanism gives order within the stream. It does not prove the real-world time at which a sensor measurement occurred. In the example, sensor and temperature are application fields, and the server-assigned ID describes append order. An application that needs observation time can store that information as a separate field.

Explicit IDs must exceed the current last ID. Repeating an old ID is an error under the candidate append implementation. This preserves ordered insertion, but it also means an application cannot insert a late observation into an arbitrary historical position merely by choosing an older timestamp-shaped identifier.

Range reads and progress use different boundaries

XRANGE handles its start and end as inclusive bounds. XREAD instead accepts the last known ID and returns entries with larger IDs. The implementation makes that distinction concrete by advancing the requested starting sequence before invoking the range-reply helper.

XREAD COUNT 10 STREAMS readings 0-0

This requests entries after the initial ID. A continuing reader uses the last ID it received in its next request. The stored data can therefore support readers at different positions, since the position belongs to the request rather than requiring destruction of the entry.

XREAD also accepts BLOCK in milliseconds. When no requested stream has eligible data, it can block the client until data becomes available or the timeout expires. The append path marks a stream ready when new data arrives, connecting insertion to blocked readers. Polling is consequently not the only way to wait for the next entry.

The special $ argument selects the current last ID when Redis processes the request. That makes it suitable for starting a wait for subsequent arrivals. Reusing $ for every request would repeatedly choose a new starting position, so a continuous reader should carry forward the concrete ID returned by the preceding read. This follows from the implementation resolving $ afresh for each invocation.

Storage groups entries without losing their identities

The low-level representation is a radix tree containing listpacks. Redis encodes IDs as big-endian numbers so lexicographic tree order agrees with numeric ID order. A tree entry points to a packed group of stream entries, reducing the need for a separate tree allocation for every application record.

Inside each listpack, a master entry records reference field names. When later entries use the same ordered field names, Redis can store their values without repeating those names. The implementation also stores entry IDs as differences from the reference ID. This is a concrete storage optimization for records with repeated shapes, such as the two readings in the example.

The representation does not require every entry in the whole stream to have identical fields. The append code checks whether the field names match the local reference entry and uses a flag when they do. An entry with different fields includes its own names. The data structure can therefore accept varying record shapes while compressing a common case.

Retention operates on this packed representation too. XADD accepts MAXLEN, and the ~ form permits approximate trimming. The trimming implementation can remove complete radix-tree nodes, which avoids rewriting a partial packed group. Approximate trimming may consequently leave more entries than the requested length. That is a meaningful difference for an application treating the count as a strict bound.

Consumer groups are still candidate code

RC1 contains consumer-group structures and commands, including pending-entry records and acknowledgement handling. XACK removes references from a group pending list and the corresponding consumer pending list. Acknowledgement therefore concerns processing bookkeeping, distinct from deleting the stream entry itself.

The same source also exposes incomplete work. In xgroupCommand, the SETID and DELGROUP branches have empty bodies in this tag. The presence of their names in help text does not establish a complete administrative implementation. A description of the first candidate must keep that source state separate from the design the project is working toward.

I regard the ordered entry model as the useful starting point for understanding this preview. The implemented append and read paths already explain why Streams is a distinct data type. The unfinished group handlers explain why the release notes ask for further candidate testing before production readiness.

The migration notes also state that Redis 5 can read earlier RDB files while earlier Redis versions cannot read the new format. That direction matters as soon as a candidate writes data files. Exploring the new data model can create persistence artifacts whose compatibility differs from the command compatibility of an existing application.