librdkafka fixes retry queues and a Windows clock overflow

A producer can remain alive and reconnecting while particular messages stop progressing toward the current partition leader. librdkafka 0.11.6 fixes a case in which retry buffers remained stuck after a connection failure. Its October 18 release notes also identify a Windows internal clock wrap that could cause stalls and hangs. Both defects matter to the continued operation of a client, beyond its ability to establish an initial connection.

The retry defect concerns messages already accepted into the client's internal work. The Windows defect concerns the time values used to schedule and expire work. Those are different causes of an apparently similar symptom, a process that no longer makes the expected progress. The maintenance release identifies and corrects both.

I value Kafka's retained log, but a record only becomes useful history after the producer delivers it. A client accepting a message locally is an intermediate state. These fixes are relevant because they concern how the library moves beyond that state during failure and prolonged operation.

Reconnecting did not release every queued message

The report in issue 1913 describes a producer sending continuously while the partition leader goes down. Some messages eventually reach message.timeout.ms and fail locally. The report's logs show repeated connection attempts to the unavailable broker while buffers remain associated with connection-reset handling.

Its reproduction uses librdkafka 0.11.5 and waits for final delivery results. The important observation is that some messages do not move to the new leader before their local timeout. The release notes describe the correction as purging and retrying buffers in the output buffer queue on connection failure.

That correction should not be described as deleting the application's messages. Purging a failed connection's request buffers belongs to the recovery path that allows their work to be retried. Leaving those buffers attached to the wrong connection state can prevent the messages from reaching another usable destination.

The 0.11.6 introduction explains the intended retry model. During recovery, the library puts retriable messages back on their partition queue, preserving their original queue positions and applying a backoff. The application receives a final delivery result when delivery succeeds or an error becomes final under the configured policy.

This separates local acceptance, internal recovery and the final result reported to the application. A connection error need not immediately fail every queued message. Conversely, continued retries do not prove delivery. The delivery report communicates the result of the library's processing for that message.

Transmission state affects what a timeout means

The introduction distinguishes a request still waiting in the transmission queue from one already in flight. Before transmission, the producer can know that the request has not reached the broker. After transmission, a missing response can leave uncertainty about whether the broker received and recorded it.

That uncertainty matters when interpreting retries. The documentation says that retrying an in-flight failure can produce duplicate messages, while retrying a request known not to have been transmitted does not have that same ambiguity. Fixing a stuck queue restores the intended recovery behavior. It does not turn every transport failure into proof that the broker never accepted the record.

I find that distinction necessary when discussing reliability. A durable broker log does not remove uncertainty at the producer's connection boundary. The client has to manage pending work and report outcomes without pretending that an absent acknowledgment answers every question about the preceding transmission.

Windows timing failed after prolonged uptime

Issue 1980 identifies a wrap in rd_clock() on Windows. Depending on the performance-counter frequency, its calculation could wrap after more than seven days of uptime. The issue says the defect was introduced in 0.11.5 and explains that timers and timeouts depend on the function behaving as a monotonic clock.

The 0.11.6 time implementation converts the performance-counter frequency to floating point before calculating microseconds. Its comment explicitly identifies avoiding overflow as the reason. The same file shows timeout initialization adding a relative duration to rd_clock() and remaining-time calculation subtracting the current clock from an absolute deadline.

Those operations explain the impact of a wrap. A scheduled deadline has meaning relative to an advancing time base. If that base unexpectedly returns a much smaller value, calculations can stop describing the intended remaining interval. A process can remain running while scheduled actions wait according to an invalid time relationship.

This failure depends on uptime and the host's counter characteristics, so a brief startup test does not reproduce its defining condition. The release repairs the conversion used by the shared timing function. Together with the queue fix, it addresses client progress both after a broker failure and after extended Windows operation, while leaving final message delivery observable through the library's existing reporting model.