March 22, 2019·4 min read·engineeringkafkasecuritysasl

Kafka 2.2 gives long-lived SASL connections an expiring session

Kafka 2.2 can reauthenticate an existing SASL connection and enforce its session lifetime. The broker setting defaults to disabled, preserving compatibility during migration.

A short-lived authentication token does not impose a short-lived connection when a server checks it only at connection establishment. Kafka's long-lived connections made that distinction consequential after OAuth bearer authentication arrived. Kafka 2.2, released March 22, adds periodic SASL reauthentication and broker enforcement of session expiry. The change lets a connection refresh its authentication without requiring a complete disconnect and reconnect.

KIP-368 describes the original problem. Once a client authenticated, its connection could outlive the token used to establish it. Disabling an account in an external directory also did not automatically invalidate that connection's authentication. Removing authorizations was the available way to stop its access. Reauthentication supplies another enforcement point for an already connected client.

I value Kafka's durable history and independent consumers, but those consumers often retain connections while they work through the log. The ability to keep reading should still depend on current authentication policy. A connection that remains open indefinitely needs an explicit rule for when its authentication must be renewed.

The broker communicates a session lifetime

The proposal increments the SaslAuthenticate API version to one and adds a session-lifetime value to the authentication response. The client can inspect the broker's supported API version to determine whether reauthentication is available. This makes the behavior negotiable across a deployment containing mixed broker versions.

The broker setting is connections.max.reauth.ms. Its default is zero, which disables expired-session enforcement and communicates no maximum lifetime. Upgraded clients do not begin periodic reauthentication merely because both sides understand the new API when the broker retains that default. The release adds a capability whose enforcement requires an explicit configuration choice.

A positive value limits the lifetime communicated to the client. The KIP gives a one-hour maximum as an example. If the presented bearer token has only 45 minutes remaining, the broker communicates 45 minutes. If its remaining lifetime is longer or unspecified, the configured maximum supplies the limit.

connections.max.reauth.ms=3600000

This example sets a one-hour maximum in milliseconds. It is a broker configuration example from the proposal's model, not a client-side token-refresh interval. Token refresh and reauthentication are related operations, but obtaining a new credential does not itself replace the credential associated with an existing connection.

Expiry enforcement happens when the connection is used

The KIP makes the enforcement condition precise. When a connection's session has expired, the broker closes it if the next request is unrelated to reauthentication. Authentication handshake and authentication requests remain permitted so the client can renew the session. An idle connection does not have to be closed at the instant its lifetime elapses.

That avoids conflating session expiry with a timer that proactively destroys every socket. The meaningful security boundary is subsequent authenticated use. A client cannot continue ordinary operations with an expired session simply because its network connection has remained established. It must successfully reauthenticate first.

On a supporting client, the networking machinery intercepts a pending send and performs the authentication exchange before allowing unrelated traffic through. Responses received while reauthentication runs can queue until it succeeds. If reauthentication fails, the broker closes the connection, and the proposal does not support retrying authentication on that same failed exchange.

This behavior has a latency consequence. The triggering request waits while the connection authenticates again. The design includes average and maximum reauthentication-latency metrics to make that cost observable. Preserving the connection avoids a new full connection setup, but the renewal exchange still performs work before normal traffic resumes.

The implementation belongs below individual client APIs

KIP-368 puts the mechanism at the network selector and channel level. Each channel tracks its communicated session expiry, and the selector starts the exchange at the relevant point in request processing. The proposal contrasts this with alternatives that inserted authentication work separately into higher-level client request queues.

That placement matters because Kafka has several users of its networking code. Producers and consumers need the behavior, but so do administrative clients and broker-to-broker uses when SASL protects those connections. Implementing renewal independently in every caller would create more places that have to coordinate the same authentication state.

The proposal also explains why killing a connection from the client alone is insufficient. A server cannot rely on a client voluntarily disconnecting when its credentials expire. Broker enforcement is necessary to make the lifetime binding. Conversely, server-side expiry without a renewal path forces legitimate clients through repeated disconnections. The adopted design supplies both renewal and enforcement together.

Non-SASL connections are outside this change. KIP-368 explicitly excludes PLAINTEXT and SSL-only connections from both reauthentication and this expired-session closure behavior. An installation cannot apply the setting as a universal connection lifetime for every security protocol.

Compatibility is an explicit part of activation

An older client cannot understand the new session information or renew its authentication through this mechanism. With positive enforcement enabled, the broker can still close that client's expired connection when it tries ordinary operations. That means enabling the setting before clients support it introduces periodic disconnection for those clients.

The proposal defines metrics for successful authentication without reauthentication support. Its migration sequence keeps the maximum at zero while brokers and clients are upgraded, then uses the metrics to identify remaining old clients before enforcement is enabled. The migration metrics also cover inter-broker connections when brokers act as SASL clients. A broker upgrade therefore has a client side as well as a server side. Retaining zero enforcement during that transition allows the new API to be negotiated without immediately requiring renewal from every participant. The setting itself is not dynamically changeable in the proposed initial delivery, so changing it requires broker restarts.

My interest in the feature is the enforcement point it creates while preserving the long-lived connection model. Authentication can become current again without redefining the stream or a consumer's position in it. The broker decides how long a session may remain valid, and subsequent ordinary requests must use a session that has met that renewal requirement.