July 26, 2021·6 min read·engineeringcassandraobservabilityreleases

Cassandra 4.0 adds node-local evidence through CQL and audit logs

Cassandra 4.0 exposes operational state through virtual tables and adds configurable audit logging. Their scope, retention rules and prepared-statement behavior determine what operators can learn from them.

An operational query needs a clear definition of where its answer comes from. Cassandra 4.0 introduces virtual tables that look like CQL tables but describe the node serving the query. It also adds audit logs stored on each node's filesystem. Both additions make database behavior easier to inspect, provided the reader preserves the local scope of the evidence.

The Apache release record dates Cassandra 4.0.0 to July 26. The release's tagged documentation describes the new virtual tables and audit logging. Those documents provide a useful account of what these interfaces report and which assumptions they do not support.

I find Cassandra attractive for applications that require active-active operation at scale. Its distributed behavior also makes the origin of an observation consequential. A local answer about clients or configuration is useful, but it cannot establish what every other node sees. The 4.0 documentation makes that distinction explicit instead of leaving the familiar table syntax to imply replication.

Virtual tables obtain data through an API

A virtual table is backed by an API rather than SSTables. Its rows represent information Cassandra exposes through that implementation. In 4.0, virtual tables are read-only, local and unreplicated. Queries ignore the requested consistency level. Increasing a query's consistency level therefore does not turn a virtual-table result into a distributed observation.

The release adds system_virtual_schema for virtual-schema information and system_views for the operational tables themselves. These are managed by Cassandra. Users cannot create their own virtual tables through ordinary DDL or modify the existing ones through DML. The shared syntax makes querying convenient while retaining a different lifecycle from a user-managed data table.

The documented restrictions follow that distinction. Virtual tables have no associated SSTables, do not support secondary indexes or materialized views, and cannot be truncated. A displayed CREATE TABLE description is a description of the exposed schema, not a statement that can recreate the implementation. The API behind the table supplies its behavior.

One practical consequence is that familiar administration habits need an accurate interpretation. A query returning a set of rows has not necessarily read replicated application data. The keyspace and table identify a different source of information. Keeping that distinction in the query itself helps an operator communicate what the result establishes.

Client connections become queryable state

The system_views.clients table describes active connections to the node. Its documented columns include client address and port, connection stage, driver name and version, protocol version, request count and TLS information. The schema provides several ways to investigate which applications are connected without treating all connections as equivalent. The documented query is straightforward:

SELECT * FROM system_views.clients;

Its result concerns the node receiving that query. The documentation suggests using the table to identify old driver versions before an upgrade, investigate clients sending many requests, or inspect TLS use during a migration. Each use depends on the columns carrying the relevant information for those connections. The sample output includes null driver fields, so absence of a reported driver name should not be converted into an invented client identity.

Connection stage and protocol version also answer different questions. A version identifies a negotiated protocol, while a stage describes the connection's state. Request counts describe activity associated with the listed connection. Reading these fields together can inform an investigation, but none of them supplies a cluster-wide inventory by itself.

This is an interface I value because the scope can remain visible in the explanation of a result. An operator can say which node reported a connection and which protocol it used. That is a more precise statement than claiming that an entire application estate has completed an upgrade based on one node's rows.

Running configuration and background work have tables

The system_views.settings table exposes current configuration settings. Its documentation names a useful case in which the YAML file has changed since startup and the operator needs the running configuration. The table also hides sensitive encryption information, including truststore details or passwords. Its output should therefore be interpreted as an operational view with deliberate omissions.

The table is read-only in this release. Querying a value does not imply that an UPDATE can change it. This distinction matters when someone turns an inspection query into automation. The documented facility supplies evidence about configuration, while configuration changes continue to need their supported administration paths.

Other virtual tables describe work inside the node. thread_pools reports active, pending and blocked tasks, along with completed-task counts and limits. sstable_tasks reports currently running operations such as compaction and upgrades. The documented fields include progress, total and a unit, which makes the interpretation of arithmetic depend on the task's measurement. For example, the documentation shows subtraction of progress from total:

SELECT total - progress AS remaining
FROM system_views.sstable_tasks;

When the task's unit is bytes, the subtraction represents remaining bytes. It does not give an estimated completion time without an additional rate assumption. The query is useful because it calculates a directly supported quantity, and its interpretation should retain that unit. A compact expression can still produce a misleading explanation if the operator silently changes what the result measures.

Audit records describe activity and authentication

Audit logging records incoming CQL command requests and authentication attempts, including failures. The documentation places the records in files on the node's local filesystem, with periodic rolling. Configuring the logger on one node does not configure every other node. Each node's settings may be the same or different. A minimal enabling fragment in cassandra.yaml is:

audit_logging_options:
  enabled: true

The default logger is BinAuditLogger, which writes binary records. Version 4.0 also supports FileAuditLogger through the standard logging mechanism. The documentation recommends the binary implementation for performance. Choosing a logger determines how the records are written and which associated logging configuration is required.

The documented event categories include query activity, data modification, schema changes and authentication. Filters can include or exclude users, keyspaces and categories. By default, some system keyspaces are excluded. An enabled audit logger with filters does not imply that every possible request appears in its retained output. The configured selection is part of any claim about coverage.

The audit facility also has defined exclusions. It does not record cassandra.yaml configuration changes or nodetool commands. DCL passwords are obscured. A file of database command records therefore cannot stand in for a complete history of every administrative action performed on the machine or through Cassandra's other interfaces.

Prepared values and retention affect the evidence

Prepared-statement execution logs the query supplied during preparation, together with the execution timestamp and other attributes. Actual bound values do not appear in the audit log. That limitation matters whenever a question requires the exact values used by an application. A statement containing placeholders is evidence of the statement form, not a record of all its execution arguments.

Retention and overload have their own controls. max_queue_weight bounds the in-memory queue of records waiting to be written. The block option determines whether logging blocks when it falls behind or drops records. max_log_size bounds retained rolled files by deleting the oldest once the size allowance is exceeded. These options trade application behavior and record retention against finite resources.

The documentation also provides roll-cycle and archive settings. The default roll cycle is hourly, while an archive command can handle rolled files and has a configured retry limit. File existence alone cannot demonstrate that every earlier record remains available. Retained history depends on the size bound and the behavior of the configured archive process.

The auditlogviewer utility reads the binary audit-log directory and produces readable output. The documented entries contain fields such as user, source address, timestamp, operation type and keyspace where applicable. Some attributes do not apply to every request, so a missing field needs interpretation in the context of its event type. The tool gives a readable representation of existing records. It cannot reconstruct values the logger omitted or events excluded by the configured filters, and it does not change the node-local origin of the files it reads.

Cassandra 4.0's new interfaces therefore support specific operational claims. Virtual tables expose local state through CQL without replicated storage. Audit files record selected node activity with defined omissions and resource limits. For an investigation involving prepared statements, the audit record retains the prepared query form but excludes its actual bound values, a distinction that remains necessary when reconstructing what a request did.