April 18, 2018·3 min read·engineeringkafkastrimzikubernetes

Strimzi improves reconciliation and advertises broker DNS names

Strimzi 0.3 strengthens its Cluster Controller and changes advertised listeners to DNS names. Its ConfigMap-driven model needs both recovery from missed notifications and stable ways to identify individual brokers.

A controller that watches configuration changes still needs a way to recover when it misses one. Strimzi 0.3's April 18 release emphasizes Cluster Controller resilience and DNS names for advertised listeners. These address separate sources of uncertainty in running Kafka on a scheduler, the controller's view of desired state and the addresses brokers publish to other participants.

The tagged controller documentation explains the configuration model. The controller watches labeled ConfigMaps describing Kafka or Connect clusters. It creates and updates Kubernetes or OpenShift resources to reflect those declarations. Periodic reconciliation compares the declarations with the running resources even when a notification was not received.

I value Kafka's durable history, but a broker deployment needs more than a successful first start to preserve access to that history. The management process must recover its understanding after interruption, and the running brokers need addresses that remain meaningful to clients. These changes concern those operational requirements directly.

Notifications and reconciliation have different jobs

A watched ConfigMap provides the desired cluster configuration through its data section. The labels strimzi.io/kind=cluster and strimzi.io/type=kafka identify a Kafka cluster declaration. Other supported types select Connect or the OpenShift Source-to-Image form of Connect. The controller creates the resources appropriate to that declaration.

The documentation describes updates as patches or deletion and recreation of managed resources. It also notes that this can cause a rolling update and service disruption. Editing a declaration is therefore an operational request, even when its delivery mechanism is a familiar Kubernetes ConfigMap update.

A watch gives the controller prompt notice of such changes. Reconciliation answers a different question, whether the current deployment agrees with the configuration now available. If the controller was stopped when an update occurred, the desired state still exists in the ConfigMap. Periodic comparison gives the controller another opportunity to act on it.

The documented default full reconciliation interval is 120,000 milliseconds. A separate operation timeout defaults to 60,000 milliseconds. These settings govern different waits. The interval controls repeated comparison, while the timeout bounds internal operations. Increasing one does not mean the other has changed.

This is the useful interpretation of resilience in the release. The controller's correctness cannot depend entirely on uninterrupted delivery of every notification. Desired state provides a durable description to compare against, and reconciliation supplies the repeated comparison. That model is inspectable even when an operator did not observe the original configuration event.

Advertised addresses belong to individual brokers

The 0.3 Kafka startup script shows the listener change precisely. It constructs advertised addresses using hostname -f. The generated configuration includes a client listener on 9092 and a replication listener on 9091, with plaintext protocol mappings for both in this script.

The important distinction is between where a process listens and the address it advertises. A wildcard listening address can accept traffic on local interfaces, while the advertised address tells another participant how to reach the broker. The script publishes a fully qualified hostname for that latter purpose.

Strimzi's getting-started guide at the same tag explains the supporting service arrangement. Regular services provide bootstrap addresses, and headless services let DNS resolve broker pods directly. StatefulSets give the pods distinct names and network identities. These pieces make a named broker address part of the deployment model.

This is not a declaration of arbitrary external client connectivity. The documented services and names concern the deployed Kubernetes or OpenShift environment. A published DNS name still needs to resolve and be reachable from the participant using it. The release's address change should not be expanded into a claim that every network outside the cluster can now reach each broker.

Defaults are controller configuration too

The release also adds configurable default Docker images in the Cluster Controller. The documentation identifies separate defaults for Kafka, ZooKeeper and Connect, with additional values for the topic controller and Connect's Source-to-Image image. A cluster declaration can still specify its own image where the corresponding field exists.

That gives the controller operator a place to establish an image default without repeating it in every declaration. The resulting deployment still depends on both inputs, the particular cluster ConfigMap and the controller's configured defaults. An omitted image field therefore has meaning relative to the controller responsible for interpreting it.

I find this relationship important when reading a cluster definition. The declaration may describe the workload's intended size and storage while relying on a management-level image choice. The April release makes that default configurable, and its DNS-based advertised listeners make the resulting brokers identifiable through the naming arrangement supplied by the deployment.