June 25, 2025·4 min read·engineeringcockroachdbobservabilityperformance

CockroachDB 25.2.2 records transient hot ranges more frequently

Qualifying ranges can now produce minute-level telemetry when hot-range logging is enabled. A new directional-access counter supplies another clue about workloads that resist useful splitting.

A hotspot that disappears before the next diagnostic sample can leave an overloaded node with little explanation in the logs. CockroachDB 25.2.2 changes hot-range telemetry to record qualifying activity more frequently. The June 25 release notes specify minute-level node checks when server.telemetry.hot_ranges_stats.enabled is enabled and a range exceeds 250 milliseconds of CPU time per second.

Multi-tenant deployments use a different scope and interval. The notes describe a cluster-level check every five minutes. Those details matter because the release does not promise one-minute records for every deployment or logging for every range regardless of activity.

I accept CockroachDB's coordination costs because its consistency model makes the reason for that work explicit. The same preference applies to diagnosis. A node-level utilization graph identifies pressure, but range-level evidence is needed to connect that pressure to a particular portion of the workload.

Conditional logging reduces the gap in historical evidence

The implementation change describes the previous regular logging interval as four hours by default. A hotspot that was not effectively continuous could go unrecorded. The new check asks whether a node is burdened by a qualifying range and logs that node's hot ranges when the condition holds.

This changes the chance of retaining evidence about a transient event. It does not turn the log into a continuous execution trace. A short event between checks can still differ from a condition observed at a check, and the threshold determines which observed activity triggers logging.

The setting can be inspected with the documented cluster-setting SQL form. This reports the configured value without enabling telemetry or changing a running cluster:

SHOW CLUSTER SETTING server.telemetry.hot_ranges_stats.enabled;

That distinction is useful when explaining missing records. A release containing the new behavior does not establish that the enabling setting was active during an incident. The configured collection behavior is part of the evidence needed to interpret the logs.

A hot range can be a moving bottleneck

The hotspot guide distinguishes hot nodes from hot ranges and moving hotspots. A range is a unit of data distribution, and concentrated access to one range can overload its hosting node while other nodes have spare capacity.

A workload appending to an ordered index can keep concentrating writes at its tail. Splitting or relocating an existing hot range does not necessarily distribute future writes, because those writes can continue following the advancing tail. The bottleneck can move between ranges and nodes without disappearing.

More frequent range records help describe that pattern over time. A single later snapshot might show a different range than the one responsible for the original latency. Historical records can preserve a sequence of observations, which is more informative than treating every newly hot node as an unrelated event.

Directional access supplies a different signal from CPU

The release adds kv.loadsplitter.cleardirection. The splitter patch records a counter and log information when sampled accesses show a strong directional bias. Its release note describes more than eighty percent of samples moving left or right, corresponding to descending or ascending access.

This signal describes the access pattern seen by the load-based splitter. It is not another measurement of total CPU utilization. A rising counter can therefore help explain why a hot range's workload is difficult to divide usefully, while CPU measurements describe the associated resource consumption.

The combination gives an operator two different questions to ask. Which range consumes the resource, and do accesses keep moving in one direction through its keys? The metric does not itself identify the application query, but it narrows the kind of key-access behavior that a query investigation needs to explain. That distinction prevents a counter increase from being mistaken for direct attribution to a particular SQL statement or application.

The diagnostic page has its own measurement scope

The Hot Ranges page reference lists range identity, key-access rates and CPU information alongside database, table and index context. Its documented rate averages are not identical to an instantaneous observation. Comparing those values with a log-trigger condition requires attention to the measurement interval.

Version 25.2.2 also moves the page's node filtering to the backend to reduce loading time. That improves the route through which an operator inspects the evidence, while the telemetry change improves what evidence can remain after a transient event.

I find these changes useful because they preserve distinct meanings for each signal. A hot-range log locates observed load, a directional counter describes sampled access, and table or index context connects the range to schema. None alone proves the cause of an incident. Together, they make a concrete workload explanation more accessible after the activity has moved or ended.