August 1, 2022·4 min read·engineeringkafkalibrdkafkaoidcauthentication

librdkafka 1.9.2 preserves the final byte of an OIDC scope request

An off-by-one formatting size could truncate the token request when a scope was configured. The patch separates the transmitted body length from the C buffer capacity needed for its terminator.

A valid OAuth client configuration can still produce the wrong request if serialization loses a byte. Librdkafka 1.9.2 fixes an OIDC POST field that was sometimes truncated by one byte. The August 1 release also updates bundled OpenSSL and Curl versions. The request fix is small in code, but it affects the identity-provider exchange that precedes Kafka authentication.

The mechanism becomes clear by comparing the request construction in the released source. The code allocates enough space for the text and its terminator, but the older formatting call does not receive the full capacity. Correct allocation alone therefore fails to preserve the intended body.

I value a Kafka client whose identity behavior can be explained through concrete configuration and protocol steps. An authentication error can otherwise send an operator toward permissions or credentials even when the client constructs a different request from the one configured.

The request has a fixed prefix and an optional scope

The 1.9.1 implementation builds a client-credentials request. Without a scope, it duplicates the fixed string grant_type=client_credentials. With a scope, it appends &scope= and the configured value, calculating the resulting body length.

For an illustrative scope value of orders, the intended body is the following text.

grant_type=client_credentials&scope=orders

The older code allocates the calculated length plus one byte. It then passes only the calculated length to rd_snprintf. A bounded C formatting operation needs room for the terminating zero byte as well as the visible text. Using the text length as the capacity leaves one fewer byte available for that text.

This explains why the release says the POST field was sometimes truncated. The branch that uses a configured scope formats a constructed string. The no-scope branch duplicates a fixed string instead. Scope length comes from the configured string, so the branch selection follows an identifiable input. A report comparing the two paths can therefore describe whether a scope was present without publishing the secret used in the authorization header. Those are different construction paths, so the defect need not appear in every OIDC request.

Body length and buffer capacity have different meanings

The 1.9.2 implementation extracts construction into rd_kafka_oidc_build_post_fields. For the scope branch, it allocates post_fields_size + 1 and passes that same larger capacity to the formatting call. The body length passed to the HTTP helper remains post_fields_size.

That separation preserves two different quantities. Buffer capacity includes the terminator required by C string handling. The transmitted form body includes the text bytes. Adding space for the first requirement does not mean appending a zero byte to the protocol payload.

The patch therefore does not need to change the intended scope or invent a new configuration option. It makes the serialized request retain the value already supplied. In the example, losing the final character could change the requested scope. The exact provider response depends on that provider, so the code alone does not establish one universal error message.

The token request happens before the broker sees a token

The source builds authorization headers from the configured client ID and secret, then sends the form to the provider's token endpoint. That HTTP helper expects JSON in response. The callback looks for a string access_token and then decodes the JWT payload for the client-side token setup.

This sequence locates the defect before the Kafka bearer-token exchange. A broker policy change cannot repair bytes already omitted from the request sent to the identity provider. Likewise, a Kafka connection being reachable does not establish that the preceding provider exchange requested the intended scope.

The implementation reports token retrieval failures separately from missing or invalid JSON fields. Those branches identify different stages of the operation. A provider response without an access_token is not the same observation as failure to retrieve a response at all.

My preference is to keep those stages visible in an incident explanation. It gives an operator a way to describe the failure without guessing that every authentication problem means the same credential is wrong.

Bundled dependency updates depend on the build

The versioned changelog specifies OpenSSL 1.1.1q for non-Windows bundled builds. Windows remains on 1.1.1n in this release. Curl advances to 7.84.0. The platform qualification is part of the release information and should remain attached to the version claim.

These are bundled dependency versions, which does not establish the linked dependency selection of every separately built application. The request-construction correction is in librdkafka itself. Its practical result is that a configured OIDC scope survives serialization with the intended text length, while the C buffer still has the extra capacity needed for termination.