Perl 5.31.8 repairs character handling at the interpreter boundary
A transliteration regression and two regex failures show why development releases need tests for combinations of language features.
Perl's transliteration operator can replace characters, delete unmatched replacements, and squeeze repeated output in a single expression. Those operations interact. In the development interpreter, a short expression that had produced one character began producing two. Perl 5.31.8 repairs that regression alongside a heap buffer over-read and an assertion failure in the regular expression engine. The common engineering problem is preserving the semantics of character operations when their internal implementations change.
The release notes shipped with 5.31.8 identify the fixes and say the development cycle has entered its contentious changes freeze. The history file in that release dates it to January 20. This is a development snapshot, so the practical audience includes module maintainers checking the interpreter against existing code. Its failures deserve precise descriptions, particularly where a general phrase such as memory corruption can obscure the actual access reported by a diagnostic.
Deletion changes which characters become adjacent
The transliteration report supplies an unusually small example. Starting with ABAB, it applies tr/AB/1/ds. The reporter records output of 1 through 5.31.6 and output of 11 in 5.31.7. A CPAN test failure for UTF8::R2 brought the difference to the reporter's attention. This is observable application behavior, even though the program remains syntactically valid and completes without an exception.
The operator documentation in this snapshot explains the underlying model. Transliteration maps characters by position between its search and replacement lists. Here A has a replacement of 1, while B has no corresponding replacement. With deletion requested, the unmatched replacement disappears. Squeezing then matters because the translated characters can become adjacent after intervening characters disappear. Treating deletion as an interruption to that relationship changes the final string.
The report's example can be written with an explicit variable so that the input and resulting value are visible:
my $text = 'ABAB';
$text =~ tr/AB/1/ds;
print $text;
The expected string is 1. The operator's return value is a separate matter, because ordinary transliteration returns the number of characters replaced or deleted. Inspecting only that number would not establish that the resulting string is correct. This distinction matters when a compact text transformation appears inside a larger expression whose caller uses the count.
I still prefer Perl for a one-off text transformation because its operators express the work directly. That preference depends on combinations of modifiers retaining their documented meaning. Replacing this expression with several separate passes would increase the amount of application code needed to reproduce behavior the interpreter already promises.
A property lookup reads past its allocation
The heap buffer report comes from fuzzing a development build with AFL and libdislocator. Its AddressSanitizer diagnostic identifies a read of one byte beyond a heap allocation in match_uniprop. The stack passes through Unicode property parsing and user-defined property handling before returning through regular expression execution. That evidence supports a claim about an invalid read. It does not establish arbitrary code execution or an attacker-controlled write.
The reporter's bisection identifies a change concerning user-defined properties overriding official Unicode properties. According to the commit description reproduced in the report, overrides had previously occurred only sometimes. Making lookup behavior consistent therefore introduced a regression in an unusual property expression. The intended semantic change and the memory error are distinct. Repairing the latter need not imply abandoning the former.
This is why a regex engine needs tests beyond familiar character classes and ordinary captures. Property names participate in resolution rules as well as character matching. An expression can exercise name handling inside a match, and a mistake there still occurs inside the interpreter process. The release note calls the fix a heap-buffer-overflow, but the original diagnostic makes the direction of access explicit. There is no basis in these sources for presenting this snapshot as a general security advisory for every installed Perl version.
Script runs expose another invalid internal state
The separate assertion report also comes from fuzzing. It combines noncapturing behavior, alternatives, empty groups, and a script-run expression. A debugging build aborts in Perl_isSCRIPT_RUN because an asserted pointer is null. The stack trace records that null argument directly. The release notes list the assertion repair separately from the property-lookup over-read, which is the correct distinction to preserve when describing the release.
An assertion checks an assumption made by the implementation. Its failure shows that the engine reached a state the implementation considered invalid. It is not a normal answer that the subject failed to match. This report therefore concerns interpreter correctness even though its reproducer looks unlike a typical log-processing expression. Fuzzing is useful here because empty constructs and alternative paths can combine in ways an example-driven test collection may omit.
My interest in these fixes is the reliability of Perl's built-in text operations. Their concise notation makes the interpreter responsible for substantial behavior that another program might spell out in loops. The transliteration report also shows how module testing complements fuzzing. One found an output regression in package tests, while the other reports isolated invalid memory access and an internal assertion. In 5.31.8, those concrete cases give maintainers specific behaviors to check against the development interpreter before depending on its character handling.