Kafka 2.1.1 corrects stale broker address state in clients

A broker's identity can remain stable while its network address changes. Kafka clients must preserve enough information to reconnect without assuming that the first resolved address remains useful indefinitely. Apache Kafka 2.1.1, released February 15, includes maintenance fixes for both DNS resolution and changes to a broker's hostname. Those fixes are a concrete reason to examine client updates separately from broker upgrades.

The release notes list KAFKA-7755 for clients resolving DNS entries only once, together with KAFKA-7890 for invalidating connection-state cache entries when a broker hostname changes. The two entries concern related but different changes in the network description. A name can resolve differently without changing its spelling, while a broker can also acquire a different name.

I value Kafka's durable log model because consumers can retain their own positions and replay history. A client that cannot reconnect to the correct broker cannot make use of that retained history. Connection-state correctness therefore belongs in the reliability story alongside replication and storage.

A node ID is not its complete network identity

The 2.1.1 connection-state implementation provides the mechanism behind these release-note entries. Its connecting method looks up state by connection ID, then compares the stored host with the host supplied for the new connection attempt. Only a matching hostname permits the existing state to be reused.

If the hostname differs, the code logs that change and creates a new NodeConnectionState. The constructor resolves the supplied hostname and initializes the address list. The cached object therefore belongs to the combination of node identity and the host information needed to reach it. Keeping the node ID alone would retain an obsolete part of that combination.

That distinction matters during a network change because a reconnect is not necessarily a repetition of the original connection attempt. New metadata can describe a different destination for the same broker identity. The client must incorporate that information before selecting an address. Retrying the old address more often would not implement the same correction.

The code also retains connection-specific state such as backoff and connection-attempt timing. A fresh state object initializes these together with the new host's addresses. The maintenance change is therefore about the validity of a cached object, rather than a global instruction to stop caching all network information.

An unchanged hostname can have changing answers

The same class stores a list of resolved addresses and a current index. When moving to another address, it increments the index modulo the list size. Returning to the beginning triggers another call to ClientUtils.resolve for the stored host and configured DNS lookup mode.

This implements a bounded reuse cycle. The client can try the addresses it knows, but finishing that cycle refreshes the information. A one-time resolution would have no such refresh point. The distinction is visible even when the hostname string never changes, because the new resolution can provide a different list.

The source is more informative than a blanket instruction to disable DNS caches. This correction defines when this client-side address list is refreshed. It does not establish that every resolver layer in a deployment has no cache, nor does it promise an instantaneous response to any DNS change. The code has a specific state transition that obtains another answer.

That scope helps explain the two release-note items together. Hostname comparison protects the meaning of a cached node entry when metadata supplies a new name. Address-list refresh protects against retaining an old answer forever when the name remains the same. Both preserve useful reuse while recognizing a condition that makes the retained information insufficient.

The maintained component is the client library

The release contains broker and other component fixes as well, but these changes live under the client implementation. Replacing broker software does not replace a library already loaded into an application process. The relevant delivered code is the client version used by the producer, consumer or other application using this network machinery.

That separation is especially relevant to Kafka's independent readers. Several applications can consume the same topic using their own client libraries and process lifecycles. Updating one reader does not establish the library version of the others. The retained records remain a shared resource, while connection handling belongs to each reader's implementation.

My preference is to preserve the log as recoverable history and keep the reader's position explicit. These fixes support that arrangement by correcting how the reader obtains a usable connection after network information changes. They do not change the partition ordering contract or the meaning of a committed consumer offset.

Kafka 2.1.1 consequently supplies a narrow but substantive maintenance improvement. A changed hostname creates new connection state, while exhausting a known address list triggers fresh resolution for an unchanged hostname. The client no longer treats either broker identity or the first DNS answer as a complete permanent description of where to connect.