Python 3.7.9 fixes hash collisions in IP interface objects

Combining integer fields with XOR can discard the information that a hash needs to preserve. The IP interface classes in Python's standard library provide a concrete example. Python 3.7.9, released August 17, includes a repair for interface objects whose hash values could collapse to the same number. The defect matters when those objects become dictionary keys or set members.

The release notes associate the change with CVE-2020-14422. Reading the actual expression explains the failure more precisely than saying that interface hashing was constant. When the interface address and network address are equal, XOR cancels them, leaving only the prefix length. Different addresses can therefore become indistinguishable to the hash calculation.

I value Python's existing networking libraries because they let application code use domain objects instead of carrying unstructured strings everywhere. That convenience relies on the ordinary object protocols working sensibly. A hash method that loses address information undermines the usefulness of an otherwise appropriate dictionary key.

Why the old expression collapses

The 3.7 branch patch shows the previous implementation. It XORed the integer interface address, the prefix length and the integer network address. No additional mixing operation separated those fields.

If an IPv4 interface describes a single-address network, its address and network address are the same integer. XOR of a value with itself is zero. The remaining prefix length is 32. The equivalent full-length IPv6 case leaves 128. This explains the constants named in the maintenance note without requiring the stronger claim that every possible interface prefix produces those numbers.

The defect follows from the relationship between the fields. Treating them as independent integers overlooked that a network address derives from the interface address and prefix. A combination rule that cancels equal values is particularly unsuitable when equality between those components is an ordinary case.

This is also why changing the address value alone does not escape the collision for those full-length interfaces. Both copies of the address change together and cancel again. The hash therefore cannot distinguish a collection of otherwise different interface objects in that case.

The replacement hashes the tuple

The released ipaddress implementation uses the same new approach for IPv4Interface and IPv6Interface. It passes the fields to Python's tuple hash operation.

def __hash__(self):
    return hash((self._ip, self._prefixlen,
                 int(self.network.network_address)))

This retains the fields as distinct tuple positions before hashing them. It avoids the explicit cancellation in the old expression. It does not promise that hashes are unique, because a hash function can still map distinct objects to the same result. The correction addresses the systematic collision caused by this particular combination rule.

The patch adds regression tests using pairs of different interface addresses. Those tests assert that the selected examples do not produce equal hashes. They target the reported constant-result failure. They do not establish a collision-free hash function over the whole address space.

The following diagnostic mirrors that narrow property for two IPv4 interfaces. Its assertions describe the intended distinction without depending on a particular numeric hash result.

from ipaddress import IPv4Interface

first = IPv4Interface("1.2.3.4")
second = IPv4Interface("2.3.4.5")
assert first != second
assert hash(first) != hash(second)

A test that only checked whether the two objects compared unequal would miss the issue. Equality and hashing are separate operations. The reported problem concerned a collection repeatedly receiving the same hash for distinct keys, not the loss of the underlying address values.

The security concern is collection behaviour

The reporter in issue 41004 connects the collisions with denial of service in programs storing interface objects in dictionaries. That is a consequence of putting these objects into a hash-based collection. Parsing one interface address does not by itself demonstrate the same collection-level cost.

The issue also records the decision to backport the correction as a security fix. That matters for Python 3.7's maintenance position. A small change in a domain class can remain relevant after normal bugfix work ends when its behaviour supplies a denial-of-service mechanism to applications using it.

I prefer using the library's interface object to inventing an application-specific representation merely to work around this defect. The shared repair addresses both interface classes and leaves callers using their existing values. Applications still need to choose the appropriate address or interface type for their own meaning.

The release itself has an unusual distribution detail. Its page explains that 3.7.8 was the last normal bugfix release, but security fixes affecting some supplied binaries justify another Windows and macOS installer update for 3.7.9. Further binary updates are not planned. The branch's security maintenance and the availability of installer artifacts therefore have different boundaries.

For this hashing change, the resulting implementation is visible in both interface classes. Address, prefix length and network address reach tuple hashing as separate fields. Full-length interface addresses no longer cancel their own address information through the old XOR expression.