June 28, 2021·4 min read·engineeringpythonhttpsecurity

Python 3.9.6 bounds discarded HTTP response headers

Python 3.9.6 applies a header-count limit while skipping a 100 Continue response. The fix closes a resource-consumption gap in a path that discards its input.

Discarding network input still requires a limit on how much input the program will discard. Python's HTTP client already limited individual header-line lengths, but its handling of 100 Continue responses could keep reading header lines without a count limit. Python 3.9.6, released on June 28, corrects that path. The important detail is that the vulnerable code was skipping headers, so it did not pass through the ordinary header parser's existing limit.

The release changelog identifies the security fix as issue 44022. Its description concerns potentially infinite reading after a 100 Continue status. Reading the implementation makes the scope more precise. The fix bounds the header block following that status, while the surrounding response-processing loop remains responsible for continuing to a non-100 response.

I value maintenance changes that expose this kind of difference between two paths through familiar library code. Python's convenience means an application can delegate considerable network behavior to the standard library. The discarded parts of a protocol still belong to the application's resource model.

The skipped block had its own loop

The 3.9.5 client implementation reads a status line in HTTPResponse.begin. When the status is 100 Continue, it enters a separate loop to skip the associated headers. Each read has a maximum line length, and an oversized line raises LineTooLong. A blank line ends the block and allows the client to read the next status.

The missing constraint is the number of lines. A peer can keep supplying lines that individually satisfy the length restriction without sending the terminating blank line. The client continues through the skip loop because no individual read violates its rule. The amount of work has no corresponding bound at the header-block level.

The normal header-parsing path already contains a count check. That creates an inconsistent resource policy inside a single response implementation. Headers that the application will use receive stronger bounds than headers the application ignores, even though both arrive from the same network peer.

This difference explains why testing only a final successful response is incomplete. The parser has already processed another block before reaching that response. An ordinary request can pass through preliminary protocol states whose resource consumption is independent of the final response's content.

The fix shares bounded reading

The 3.9.6 implementation extracts _read_headers from the ordinary parser. The helper reads potential header lines, checks each line's length, and rejects a block when its collected line count exceeds _MAXHEADERS. Both parse_headers and the 100 Continue skip path use it.

The constants in this version are _MAXLINE = 65536 and _MAXHEADERS = 100. These are implementation details, not a new application configuration interface. The helper counts collected lines, including the terminating blank line, so describing the limit as exactly one hundred meaningful header fields would be imprecise.

The refactoring also separates reading from interpretation. parse_headers joins and decodes the collected bytes before using the email parser. The continuation path only needs to read the bounded block and discard it. Sharing _read_headers applies the resource checks without requiring a skipped block to become an application-visible header object.

That approach reduces duplicated protocol handling. If the two paths retained independent read loops, a future change to one limit could leave the other path behind again. The shared helper states the relevant invariant in one implementation while allowing callers to use or discard the returned data according to their role.

A timeout answers a different question

The defect concerns continued progress through valid-sized lines. A per-operation timeout does not by itself describe the maximum number of lines a peer can send. The implementation's count limit supplies a different bound, which remains relevant when each individual read succeeds.

The patch is correspondingly specific. _read_headers enforces its limits for each invocation. It does not add a limit on how many separate 100 Continue responses the outer loop can encounter. Those are different protocol quantities, so the presence of a header-count check should not be described as a complete bound on every possible response sequence.

A caller using http.client still needs to handle a response-processing exception. After an invalid block exceeds the library's limit, treating the operation as a normal completed response would conceal the failure. The useful outcome of the patch is that the client can stop processing that malformed block and report an error.

I prefer the shared reader because it makes the resource rule independent of whether the application intends to keep the headers. That is an inspectable implementation decision with a direct consequence for the amount of work permitted in this path.

Python 3.9.6 gives the continuation-header path the same bounded reading mechanism used for ordinary headers. The change affects applications whose HTTP interactions reach that path, including interactions with a peer that never supplies its terminating blank line. The parser can now reject an excessive block based on its accumulated line count even when every line remains below the individual length limit.