June 27, 2020·4 min read·engineeringpythonopensslconcurrency

Python 3.7.8 rejects OpenSSL builds without thread support

A hang in the SSL tests exposed an unsupported OpenSSL configuration. Python 3.7.8 makes the thread-safety requirement explicit when compiling its extension modules.

A successful compilation can conceal a dependency configuration that the interpreter cannot use safely. Python 3.7.8, released June 27, closes one such gap by requiring OpenSSL thread support when building the SSL and OpenSSL-backed hashing modules. The change turns an implicit assumption into a preprocessor check with an explicit error.

The release notes explain the underlying compatibility boundary. Python 3.7 made thread support mandatory, and a non-threaded OpenSSL build no longer works safely with it. This maintenance change does not introduce that Python requirement. It detects a dependency configuration inconsistent with the requirement.

I value the capabilities that Python exposes through native libraries, but those dependencies deserve as much attention as the Python modules importing them. A Python version alone cannot describe the properties of a separately configured cryptographic library. Rejecting an unsafe configuration during compilation makes that missing condition concrete.

The report began with a hang

The reporter in issue 40515 had built Python 3.7.7 and OpenSSL from source on CentOS and observed hangs in the SSL tests. OpenSSL had been configured with its no-threads option. The proposed local change adjusted the condition around a counter used in Python's SSL thread-handling code.

That proposal addressed an observed symptom, but the discussion identified a broader requirement. Christian Heimes explained that Python requires thread-safe libraries and that supporting a non-threaded OpenSSL would demand extensive, careful locking in the integration code. He warned that the unsupported configuration could cause memory corruption and eventually crash the interpreter.

The distinction is important when judging a proposed concurrency fix. Making a particular test stop hanging does not establish that every use of a library's shared state is safe. The report supplied useful evidence of a bad configuration. The accepted repair enforced the configuration contract instead of claiming that the small local adjustment supplied all necessary synchronization.

This is a dependency-level decision. The application cannot infer the safety of the native library merely from the fact that its Python wrapper imports or completes a sample operation. The build needs to establish the property required by the wrapper's implementation.

The check applies to SSL and hashing

The 3.7 backport adds the same compile-time guard to Modules/_ssl.c and Modules/_hashopenssl.c. The latter implements the OpenSSL-backed hashing module. Checking both locations prevents this requirement from being expressed only at the TLS wrapper.

The added guard is short enough to show in full. Its error describes the missing dependency property directly.

#ifndef OPENSSL_THREADS
#  error "OPENSSL_THREADS is not defined, Python requires thread-safe OpenSSL"
#endif

This is a compiler preprocessor condition, not an application exception or a runtime lock. A build using headers without OPENSSL_THREADS cannot compile those guarded modules successfully. That is the intended rejection. Removing the check would remove the diagnostic while leaving the requirement described in the issue unresolved.

The patch also simplifies a conditional in the SSL module's initialisation. Previously the OpenSSL 1.1 path checked both the version condition and the thread-support macro before recording built-in thread support. After the new guard establishes that the macro is defined, the later branch can test the OpenSSL version condition alone.

The two edits therefore relate to the same assumption at different locations. One establishes the prerequisite during compilation. The other removes a redundant condition from the initialization path once that prerequisite has been enforced. Neither should be read as a general implementation of locking around a non-threaded OpenSSL.

The build configuration is part of compatibility

The immediate audience is people compiling Python against their own OpenSSL build. The reporter's no-threads option changed the relevant property even though the library still belonged to the expected OpenSSL release family. A version number and a configuration choice describe different aspects of the dependency.

For a distributor, the new error supplies a reason to examine how the dependency was configured. It is more informative than debugging a later hang through Python-level SSL calls. The accepted change also defines the supported direction. The OpenSSL build must supply thread support for these modules.

I prefer a build that refuses a configuration it cannot support to an artifact whose limitations only become visible under concurrent use. That judgement concerns a specific invariant here. It does not require treating every non-default build option as a defect or assuming that every native dependency has the same threading model.

The issue's evidence is correspondingly bounded. It records a test hang with a particular custom configuration and a maintainer's explanation of the broader safety problem. The patch verifies the presence of the thread-support macro in the dependency headers. It does not claim to test every property of an installed OpenSSL library or every application's concurrency behaviour.

For Python 3.7.8 builders, the resulting requirement is straightforward. Both native modules check OPENSSL_THREADS before compiling. A dependency build that omits it now produces a message saying that Python requires thread-safe OpenSSL, placing the failure at the configuration boundary that the original hang had exposed.