December 6, 2022·4 min read·engineeringpythonloggingsecurity

Python 3.11.1 escapes control characters in HTTP server logs

Python 3.11.1 changes how http.server writes request data to stderr. Escaping control characters and backslashes makes the log safer to display and less ambiguous to interpret.

Request data can reach a terminal through an error log even when a server rejects the request itself. Python 3.11.1 addresses that route in http.server. Its default logging method now escapes control characters before writing the message to standard error, so malformed request data does not reach the terminal in its original control-character form.

The December 6 release is the first maintenance release of Python 3.11. Its changelog identifies the logging change as a security repair. A related entry explains why backslashes must be escaped as well. The implementation needs to preserve the difference between an escape it generated and similar-looking characters that arrived in the request.

I value small standard-library utilities when their behavior is explicit. A log is part of that behavior. Rejecting a malformed HTTP request does not end the server's responsibility for the request bytes if it then displays those bytes to the person operating the process.

Rejected input still has a destination

The repair concerns BaseHTTPRequestHandler.log_message, the common method used for logging in the server implementation. The changelog describes terminal control characters arriving in a garbage request and being printed to the server's standard-error log. That output path can exist independently of whether a request reaches an application handler.

The important boundary is therefore the log destination. A terminal consumes character sequences according to its own display behavior. Bytes that were merely invalid HTTP input can retain control significance when another component prints them. Validation for the network protocol does not automatically encode the data for that later consumer.

Python's fix changes the representation at the point of output. It retains a textual record of the character while preventing the original control character from appearing in the message. That allows an operator to inspect malformed input without requiring the terminal to interpret the input's raw controls.

This is a more specific operation than deleting suspicious characters. Deletion would remove evidence about what the server received. A visible hexadecimal escape preserves an identifiable representation, which is useful when the malformed part of the request is itself the reason for the log entry.

Translation happens after message formatting

The 3.11.1 server source builds a translation mapping for characters below hexadecimal 20 and the range beginning at hexadecimal 7f and ending before a0. It also adds a mapping for the backslash. The mapping converts the selected controls to visible hexadecimal escape sequences.

Inside log_message, the implementation first applies the supplied format to its arguments. It then translates that completed message before passing it to sys.stderr.write. That ordering matters because request-derived characters can arrive through the format arguments. Translating the format string alone would leave those inserted values untouched.

The method continues to include the client address and current date and time in the surrounding log entry. The repair concerns how the formatted message is represented. It does not require every caller to change its logging format or replace its request-reporting operation.

The translation is also local to this output method. It is not a promise that every use of request data throughout Python receives the same encoding. A custom handler that overrides logging owns the behavior of its replacement, including the representation it sends to a terminal or another destination.

Backslashes make the record recoverable

Escaping a control character introduces a visible sequence beginning with a backslash. Without another rule, a request containing those literal printable characters could produce the same displayed text. The log would then leave readers unable to tell whether the server saw a control character or an ordinary backslash followed by a hexadecimal-looking suffix.

The changelog explicitly addresses this ambiguity. Escaping backslashes makes it technically possible to parse the logged representation and reconstruct the original message data. It distinguishes characters supplied by the client from syntax added by the logging transformation.

This is an information-preservation requirement as well as a display requirement. Two distinct inputs should remain distinguishable after encoding if the log is meant to explain what happened. A repair that only prevented terminal effects could still discard that distinction.

I prefer the explicit escaping rule for this reason. It preserves evidence about the malformed input while giving the terminal a printable representation. That is a useful property for a diagnostic interface, even when the server itself is a small development utility.

The first maintenance release also contains repairs to exception-group handling and traceback locations, but this logging change addresses a different part of failure reporting. It controls how untrusted message content crosses into the operator's display. The completed log message now passes through a defined translation, with escaped backslashes retaining the distinction between original text and the control-character escapes the server adds.