Perl 5.25.8 chooses its hash function by key length

A hash function pays for every byte of its input, but the shape of that cost matters. A loop that is reasonable for a long string can spend an unnecessary part of its work controlling iteration over a short key. Perl 5.25.8 addresses that difference with a hybrid default on 64-bit builds. Its release notes describe an optimized One At A Time Hard variant for keys of sixteen bytes or fewer, followed by SipHash 1-3 for longer keys. The release history records December 20, 2016.

The change interests me because Perl's appeal for one-off text work depends on operations below the level of the script. A program can keep the same hash expressions while the interpreter changes how it processes their keys. That gives the implementation room to improve common operations without adding a new choice to every caller. The useful explanation lies in the dispatch and the two implementations, not a general promise that every script becomes faster.

A bounded path for short strings

The versioned hash implementation makes the boundary explicit. Its hybrid function switches on the input length. Cases for sixteen bytes down to one consume a known number of bytes, with execution falling through the cases. The source comment explains that this optimized variant reads the key in reverse order to avoid loop overhead.

That structure replaces a repeated loop condition with a single length dispatch for these bounded inputs. Each case performs additions, shifts, and exclusive-or operations before reading its selected byte. The length fixes how many cases execute. This is a specialization for a small set of input sizes, so the implementation can spell out the work that a general loop would otherwise repeat.

The older One At A Time Hard function remains in the same header. It initializes its state using seed material and the input length, then consumes bytes through a loop. Comparing the two implementations gives a concrete reason for the short-key optimization. The hybrid does not merely call that existing function under a new name. Its dedicated short path changes the organization of the byte processing.

Sixteen is a byte boundary in this code. It is not a count of words or displayed characters. The function accepts a pointer and a length, and dispatches on that length. Describing the rule as a short-name optimization would hide the relevant input. What matters here is the size of the byte sequence passed to the hash function.

Longer keys use SipHash 1-3

For inputs beyond the short cases, the hybrid selects SipHash 1-3. The header includes implementations of both SipHash 1-3 and SipHash 2-4, generated from a shared function definition. The parameters select how many rounds run while processing the input and during finalization. In the 1-3 instance, the source supplies one round for the former and three for the latter.

The implementation processes full eight-byte input blocks, then assembles the remaining bytes into a final block that also includes length information. Its state uses 64-bit values. At the end, this Perl implementation returns a 32-bit result from the combined state. The width of the internal arithmetic and the width of the returned hash are therefore separate details.

The release notes call the long-key change a large performance improvement and describe the short-key gain as modest. They give no numerical speedup in that section. Those statements concern the hashing work, and they do not supply a numerical claim about an entire application. A text-processing program still performs its other operations. The practical contribution is that the interpreter now makes a length-sensitive choice within an operation the program already uses.

The default depends on the build

The header selects the hybrid when no explicit supported hash-function macro has been defined and IVSIZE equals eight. Otherwise, its default is One At A Time Hard. This makes the release notes' qualification about 64-bit builds concrete. The default comes from interpreter configuration and compilation, rather than a branch inserted into each Perl script.

Seed handling is visible in the same definitions. The hybrid declares twenty-four seed bytes, while the standalone SipHash definitions declare sixteen. A common macro passes the chosen seed and key to the configured implementation. The header's introductory comment identifies resistance to algorithmic complexity attacks as the purpose of its seed-based hashing support. That context explains why the release notes discuss security as well as performance when describing hash changes.

I prefer this level of specificity to treating a hash function's name as a complete security argument. The release supplies a particular combination of functions and a particular dispatch rule. It does not establish that arbitrary input is harmless, nor does the performance statement quantify resistance to an attack. Reading the implementation keeps those claims at the level supported by the sources.

Fewer experimental build choices

This release also removes several nonstandard hash-function build options. The release notes explain that these had existed for testing since Perl 5.18 and were not recommended. The removed options include SDBM, DJB2, and several Murmur variants. That is a separate change from the new default, but it concerns the same maintenance problem. An interpreter can support experimental configurations without recommending them for ordinary use, and continuing to carry those configurations has a cost.

For someone building Perl, the remaining definitions in the header state what implementations this release supports. For someone writing a script, the useful model is simpler. Key processing remains an interpreter responsibility, and the default 64-bit implementation now treats short and long inputs differently. The source makes that division precise enough to explain the performance claim without inventing an application benchmark or requiring a new language construct.