Metis, Logos & Arachne

Why it exists

Loki, Mimir, and Tempo use three unrelated storage models. Loki stores logs in compressed chunks with a separate label index, Mimir stores metrics in TSDB blocks managed by a shipping, compaction, and store-gateway pipeline, and Tempo stores traces in Parquet blocks with a dedicated WAL and block builders. An operator who runs all three must maintain three distinct models of data location and failure behavior. This complexity results from using a bespoke storage engine for each signal rather than from the underlying problem.

The read path has a structural limitation as well. Loki's index maps labels to chunks but contains no information about their contents. A line filter must therefore fetch, decompress, and scan every chunk in the relevant range even if only a small fraction of lines match, and twice the volume requires reading twice the number of chunks. In addition, accumulated dashboards, alerts, and recording rules written in PromQL, LogQL, and TraceQL account for most of the cost of leaving an established observability platform. A replacement that requires those assets to be rewritten is therefore unlikely to be adopted, irrespective of its merits.

What it is

Metis, Logos, and Arachne are three observability backends based on the principle that the query language is the contract while the storage engine is an implementation detail. Metis accepts Prometheus remote-write and OTLP and serves PromQL in place of Mimir. Logos provides Loki compatibility across more than 70 HTTP endpoints and serves LogQL. Arachne ingests OTLP, Jaeger, and Zipkin spans and serves a documented subset of TraceQL. Each system stores its data in ClickHouse, so Grafana, Alloy, Promtail, and the OTel Collector can connect after only an endpoint URL is changed.

The three systems are deliberately isomorphic. They use the same single-binary layout with a -target flag for running either one component or all components, the same Kafka-backed write path, the same tenant-as-a-column model, the same Valkey caching, and the same middleware chain. Consequently, operational knowledge acquired from one system applies to the others. Their maturity remains uneven. Metis is production-grade, has 61 documented HTTP endpoints, and has a pure-Rust reimplementation underway. Logos is implemented against a full architecture RFC and has a published performance analysis, while Arachne is fully specified but has not yet been implemented.

How it works

Each system functions as a compatibility overlay. A parser translates PromQL, LogQL, or TraceQL into parameterised ClickHouse SQL, while a residual executor processes constructs that SQL cannot express. Columnar reads, per-column codecs, bloom-filter indexes, and partition pruning take the place of whole-chunk scans. Several subsystems in the incumbent designs have no corresponding component in these systems, including chunk lifecycles, block shipping, store gateways, WALs, and compactor fleets. MergeTree parts, TTL clauses, and native storage policies perform their functions, which removes the associated operational surface.

Metis provides an integrated SLO engine and rollup tables at several resolutions, and it selects the tier deterministically from the query step. Logos assigns dashboard queries, full-text scans, and alert evaluation to physically separate paths. As a result, a long scan initiated during an incident cannot delay the alert evaluation running at the same time. Arachne adds a coarse time-bucket marker to every span, allowing inexpensive trace-by-ID lookup without prior knowledge of when the trace occurred. Its metrics generator derives RED metrics and service graphs from the span stream and remote-writes them to Metis through the standard ingest path.

Performance

The performance rationale is that ClickHouse was designed for exactly the workload that the incumbent systems approximate. Because storage is columnar, a query that requires only the timestamp and line columns reads only those columns and omits all label data. Vectorised execution processes column batches with SIMD instructions instead of handling one row at a time. Compression codecs are selected separately for each column to exploit homogeneous types and cardinalities. These codecs include delta encoding for timestamps, LZ4 or ZSTD for payloads, and LowCardinality dictionary encoding, which stores a column with only a few distinct values, such as a log level, at nearly zero marginal bytes per row. Interleaved chunk formats cannot use these techniques, so columnar layouts attain compression ratios for log data well beyond those of chunk stores. For reads, bloom-filter, token-bloom, and set indexes exclude granules of roughly 8,192 rows without scanning them, while time-range and tenant filters omit entire partitions. A content filter therefore reads only the matching granules from the referenced columns, whereas Loki must decompress and scan every line in every chunk within the range.

The project's performance goals are expressed as properties rather than as benchmark figures. The requirement that ingested data be queryable in under a second is met by a dual-write to a short-TTL buffer table, which is merged with the main table at read time instead of waiting for part merges. Predictable query latency under incident load is enforced by three-tier routing, which places dashboard queries, needle-in-haystack scans, and alert evaluation on physically separate paths and assigns the slow tier to an isolated read replica. The requirement that wide-range queries not slow down as the range grows is met by rollup tiers at raw, one-minute, five-minute, and one-hour resolution; the tier is selected deterministically from the query step. An inexpensive retention horizon is provided by per-column compression and native tiered storage across NVMe, object storage, and archival object storage. Logos's published execution analysis defines the overall target under which, even in worst-case hybrid execution where some pipeline stages must run in Go, line-filter push-down, columnar I/O, and partition pruning are expected to keep the new architecture's performance floor above the old architecture's ceiling.

Why it matters

When all three signals use one engine and a common schema convention, cross-signal correlation becomes a SQL join rather than a UI integration feature. Raw analytical SQL remains available beneath the compatible query languages. This access supports analyses that the incumbent languages cannot express and provides a workable exit path to any customer with SQL access to its own data. Maintaining compatibility at the query layer also keeps a migration reversible at a bounded cost throughout its duration. The SQL surface further enables machine-driven analysis because an agent cannot operate a dashboard but can write SQL against live telemetry without first exporting the data.