August 20, 2019·4 min read·engineeringperlcompatibilityoperators

Perl corrects ranges beginning with the string zero

Perl 5.31.3 treats the plain string zero numerically in list ranges. A special case intended to preserve padded strings had produced an unexpected sequence through 99.

A range beginning at the string "0" and ending at "-1" used to produce numeric strings through "99". Perl 5.31.3 makes that range empty, matching the corresponding numeric expression. The release notes explain that the surprising result came from a special case for strings starting with zero. That special case had a useful purpose, but it had failed to distinguish a plain zero from a padded string.

The August 20 development release corrects that distinction without describing every quoted range as numeric. Padded strings still need their string behavior. The specific repair gives the single-character string "0" the numeric interpretation needed for an ordinary descending range to be empty.

Perl's concise operators are part of why I still reach for it for one-off text work. They also require careful reading when an operator has several interpretations. This correction is interesting because the problem is neither a new operator nor an exotic value. It is the familiar zero taking an unintended path through an existing special case.

Padding explains the original exception

The release notes give the reason for treating some strings beginning with zero as strings. A range such as "00" through "03" should preserve the padding in its generated values. Turning both operands into numbers would lose that formatting. The documented behavior produces "00", "01", "02" and "03".

The problem was extending that treatment to the plain string "0". That value contains no additional padding to preserve. In the reported descending case, string-range behavior generated values through "99" instead of producing the empty list expected from zero through negative one.

These three expressions make the distinction concrete.

my @numeric = (0 .. -1);
my @plain_zero = ("0" .. "-1");
my @padded = ("00" .. "03");

In 5.31.3, the first two arrays are empty. The third contains the padded strings described above. The examples deliberately place the operator in list context, because that context is essential to the behavior being discussed.

This change affects the sequence an expression generates, not just how a debugger prints its operands. A caller iterating over the earlier unexpected result could execute its loop for values it did not intend to generate. Making the result empty for the corrected case restores the numeric boundary implied by the two endpoints.

Context selects a different operator

The operator manual included with 5.31.3 describes two meanings for binary dots. In list context, the range operator generates values counting upward between its endpoints. A left endpoint greater than the right endpoint yields an empty list for the numeric case. String ranges use the language's automatic string increment behavior.

In scalar context, the same spelling is a stateful boolean operator. It stays false until its left condition succeeds, then remains true until its right condition succeeds. The manual relates that behavior to line selection in text-processing tools. It does not generate a list for the scalar caller.

That distinction limits what the August correction means. Assigning a range to a scalar is not a way to count the values of the list example above. It selects different semantics. Explaining the bug without naming list context can therefore leave a reader with a test that exercises the wrong operator.

The data representation also matters inside list context. Preserving "00" is a string-formatting requirement. Interpreting "0" numerically is the corrected boundary case. Neither rule implies that quoting arbitrary endpoints forces the same behavior for every range expression.

My preference is to make that context apparent in examples. An array assignment shows what is being generated without relying on a reader to infer it from a larger expression. That small amount of surrounding code makes a description of this change substantially less ambiguous.

Regex combinations lose an undefined case

The same release prohibits \K inside lookahead and lookbehind assertions. The release notes say the combination caused unexpected behavior and that nobody could define the desired behavior. This is a separate incompatible change, but it concerns a similar problem of combining individually useful language rules without a coherent result.

The keep operator changes which part of already matched text belongs to the final match. A lookaround assertion imposes a condition without becoming consumed match text. Nesting the former inside the latter had left an unclear relationship between those behaviors. Perl 5.31.3 rejects the combination instead of retaining an interpretation that the project could not specify.

That restriction does not describe a general removal of \K. Its scope is the operator's use inside lookaround assertions. Keeping that scope explicit matters for existing substitution expressions that use the operator outside assertions. A prohibition on one combination should not be reported as the disappearance of the underlying facility.

Internal maintenance has measurable scope

The release also reduces memory bookkeeping overhead for an OP structure by eight bytes per operation on 64-bit systems. The notes give that unit explicitly. It is not a claim that every program's total memory use falls by eight bytes per source expression, because source text and allocated operation structures are different quantities.

An exception-handling correction prevents infinite recursion when Perl tries to update a read-only $@ while reporting another exception. The interpreter now replaces that value with a copy if it is not a simple writable scalar. The intended result is to avoid an exception during error reporting repeatedly triggering the same path.

Those changes have their own contracts. The range repair remains a narrowly described change to the plain string zero in list ranges. Padded values keep their useful string interpretation, while a descending range beginning with "0" now follows the same empty-result rule as its numeric counterpart.