Perl 5.29.4 expands regex counts below an internal sentinel

An implementation limit can be easy to misstate when its largest stored value also has a special meaning. The Perl 5.29.4 release notes describe a doubled pattern repetition limit of 65,535. Reading the compiler gives a more precise account for explicit counted quantifiers. The compiler rejects counts greater than or equal to REG_INFTY, while the normal definition of that constant is 65,535. The highest explicit accepted count is therefore 65,534 under that definition.

Aaron Crane announced the development release on October 20. The increase is real, but distinguishing an explicit finite count from the internal representation of an unbounded count matters when writing a pattern near the limit. A release-note headline alone does not explain that boundary.

I favour Perl for direct text processing, and counted repetition is a useful part of that expressiveness. The exact accepted bound matters more than a rounded description when an application constructs patterns from lengths. A generated quantifier must satisfy the parser rule, regardless of how the release summary describes its capacity.

The parser reserves the top value

The compiler at the release tag parses the numeric parts of counted quantifiers with grok_atoUV. For both the lower and upper explicit counts, it checks whether the parsed value is at least REG_INFTY. If so, compilation fails with a diagnostic stating that the quantifier is larger than REG_INFTY - 1.

The constant definition in regcomp.h normally uses U16_MAX. It also supplies a 16-bit-sized default for platforms where a short integer is wider than two bytes. With the default value of 65,535, subtracting one gives the explicit finite limit of 65,534.

The same compiler compares a maximum count to REG_INFTY when representing an unbounded repetition. Debug output prints that value as INFTY. These uses explain why the largest internal value cannot simply be offered to a user as another ordinary finite count. It conveys a distinct condition in the compiled representation.

A pattern that needs a finite count near the boundary therefore needs the finite rule. A pattern using an unbounded quantifier expresses a different requirement. Substituting one for the other merely to avoid the count check would change the accepted input, unless the rest of the pattern independently constrained the length in the intended way.

This also explains why a claim about a+ should not be copied into advice about exact counts. The release notes use an example involving that quantifier, but the compiler distinguishes unbounded maxima from explicit numeric bounds. The source provides the precise rule needed for generated {n,m} syntax.

Compilation no longer starts with a sizing pass

The release notes identify another substantial regex change. The compiler no longer performs its sizing pass, though some patterns may need an extra pass to count parenthesized capture groups. This changes the compilation procedure beneath the pattern syntax. It does not give application code a new matching operator.

That change belongs to the same compiler whose internal count representation determines the quantifier limit. Internal restructuring and user-visible capacity can arrive together while having different compatibility implications. An application sees accepted syntax and matching behaviour, while the compiler can change how it constructs the engine representation for that syntax.

The notes also report a repair to script runs that had failed to permit ASCII digits in some cases. This is another reason to keep claims specific. The release expands a count limit, changes a compilation pass, and fixes a script-run matching case. None of those descriptions establishes a general speedup for every pattern or a new guarantee about arbitrary regex workloads.

Other operations become explicit errors

The release makes previously deprecated raw I/O on :utf8 handles fatal. The affected functions are sysread, syswrite, send, and recv. According to the release notes, these functions paid attention to the UTF-8 flag while ignoring other layers. A handle with a UTF-16LE encoding layer could consequently receive UTF-8 treatment.

For input, the functions also bypassed layer validation without supplying equivalent validation of their own. That could produce invalidly encoded Perl scalars. Rejecting the calls prevents an ambiguous interaction between byte-level operations and an encoding-marked handle. It is a separate compatibility change from the regex count increase.

I consider the common requirement to be precise about representation. An encoded handle is not adequately described by one flag, and a quantifier limit is not adequately described by the largest integer in its storage field. Perl 5.29.4 documents the I/O rejection, while its compiler source supplies the finite-count condition needed to interpret the repetition increase accurately.