Confluent's Bellevue workshop teaches continuous SQL over Kafka
A February Kafka and KSQL session introduces streaming SQL as a continuously running computation. Its useful distinction is between an event stream and an evolving table, even when both originate in Kafka topics.
SQL over a stream needs an explanation of when the query finishes and what a new record means for its result. Confluent's February 23 knowledge-series event in Bellevue puts that question into its introduction to Kafka, the Streams API and KSQL. The announced agenda covers the architecture and deployment of continuous queries, with streaming ETL and analytics among its stated subjects.
The event is an educational session, not a broker release. Its published agenda gives an overview followed by Confluent's presentation and a question period. It does not establish that particular attendees built or deployed an application. The useful technical subject is the processing model the session proposes to teach.
I find Kafka most interesting when its retained history lets several consumers derive their own results. Streaming SQL makes one of those consumers expressible as a query. The important question remains what the derived result represents and how each new event changes it.
A continuous query keeps processing new input
The original KSQL introduction, published in August 2017, describes continuous transformations over Kafka topics. The query remains active as new records arrive. That differs from a finite database query whose execution returns a result and completes.
Its architecture separates the interactive client from server processes executing the queries. The client sends commands over a REST API and can inspect or terminate running queries. Kafka Streams supplies the underlying processing library. SQL is therefore an interface to a running processing application, with a server responsible for its execution.
This affects the meaning of submitting a statement. The user needs to understand the lifetime of the resulting computation, rather than assuming that disconnecting a terminal describes the lifetime of every query it submitted. The architectural separation between client and server is what makes the distinction relevant.
The KSQL 0.3 README identifies that release as a developer preview and explicitly excludes production-cluster use. Its examples nevertheless provide a concrete way to learn the model before adopting it for an operational service. A workshop can be useful for understanding that model without implying that preview status has changed.
Streams and tables interpret records differently
The launch description distinguishes immutable events in a stream from evolving facts in a table. A page view is naturally an event. A record giving the latest information for a user can instead update the state associated with that user's key. Kafka topics can carry either interpretation, depending on what the application intends each record to mean.
This distinction determines the query's result. Counting independent events asks how many things happened. Maintaining a table asks what information currently applies to each represented entity. A SQL expression does not remove the need to choose between those meanings before processing begins.
The 0.3 README's anomaly-detection example groups authorization attempts by card number over a five-second tumbling window and filters groups whose count exceeds three. Its threshold and window are illustrative. The example's value is that it makes the relationship between grouping, time and output explicit in one definition.
This windowed computation is not an ordinary count over all events ever seen. Its time boundary is part of the definition. A reader inspecting the query can identify which observations belong to the same calculation and which key distinguishes one calculation from another.
I regard that key choice as a substantive design decision. Kafka's partition-local ordering already requires care about which records belong together. A shorter query does not eliminate the need to explain the entity being counted or the period over which its events have meaning.
The surrounding tools make the computation inspectable
Confluent's December 2017 KSQL update adds Avro support through Schema Registry. It describes inferring columns from a topic's registered schema and registering output schemas when needed. That makes data representation part of the query environment, with compatibility settings still governing registered schemas.
The same update adds statistics through DESCRIBE EXTENDED and information through EXPLAIN, including execution plans and processing rates. Those commands help distinguish a declared query from one that is actually processing the intended records. Serialization failures are also part of the reported information, which matters when a topic exists but its records do not match the expected representation.
The November preview update had announced monthly releases and a beta program. That history gives the February session a specific context. KSQL is developing both a query interface and the operational tooling needed to understand its execution.
The Bellevue agenda offers an introduction to that combination. A continuous query expresses a transformation that remains active, while a table gives a particular interpretation to a sequence of updates. Observability and schema handling make those definitions easier to inspect as programs that process records. The educational value lies in connecting the familiar SQL notation to those concrete execution semantics.