Perl repairs premature scope unwinding inside regex matches

A regex engine that supports embedded code must track more than the position in the input string. It also has to restore temporary program state at the correct point in matching and backtracking. Perl 5.43.6 fixes an interaction that could restore that state too early when a pattern combined a subpattern with an atomic cut.

The release notes identify subpattern forms such as (??{...}) and (?&FOO), together with an atomic group written as (?>...). Under some combinations, a localization inside (?{ local $x = ... }) could unwind before matching finished. The history records date the development release to December 20.

Perl's integrated regex syntax is still one of the reasons I choose it for text processing. This repair concerns the more complex end of that facility, where a match interacts with program state. The important behavior is the lifetime of a localized value, rather than the amount of text a pattern accepts.

Embedded code has a matching lifetime

The versioned regex reference draws a precise distinction for embedded code blocks. A block introduces a lexical scope for declarations, but it does not end the lifetime of local changes when the block itself ends. Later code blocks in the same pattern can still observe those localized values.

The accumulated localizations unwind when the match succeeds or when backtracking passes the relevant assertion. This ties temporary state to the engine's path through the match. A value can need to remain available after the code block that set it has returned, because matching has not yet abandoned that path.

The distinction matters because lexical declaration and dynamic localization solve different lifetime problems. A reader treating every closing brace as the end of every temporary value would misunderstand the documented regex behavior. The engine's state restoration has to follow the matching operation's rules for local.

The reference demonstrates this with a counter updated through localizations while a pattern consumes repeated characters. Backtracking unwinds the counter to the value corresponding to the surviving match path. That example explains why restoring state too early changes observable results even if the engine still reports a successful match.

Atomic groups restrict one kind of backtracking

An atomic group matches what its internal pattern selects at the current position, then prevents backtracking into that completed group. The reference also emphasizes a limit to that restriction. Matching can still backtrack past the entire construct. Atomicity does not eliminate the surrounding engine's state transitions.

The documented example ^(?>a*)ab illustrates the difference. The atomic a* consumes the available initial a characters and will not return one to satisfy the following ab. A non-atomic repetition can return characters as the engine searches for a successful continuation.

That restriction creates additional bookkeeping when combined with subpattern invocation and embedded code. Some paths become unavailable for further exploration, but state associated with the continuing match still needs its correct lifetime. Removing a backtracking choice does not authorize restoring unrelated localized values before their intended use has ended.

The defect fixed in this release is therefore more specific than an atomic group matching the wrong substring. The release notes describe premature scope exit in other code during the match. A variable localized by a code assertion could lose its temporary value while matching was still in progress.

Subpatterns add another control-flow boundary

The reference defines (?&NAME) as recursion to a named subpattern. It also describes (??{...}) as a postponed subexpression whose code result supplies a pattern. These forms involve matching work beyond a simple linear sequence of literal tokens.

The release notes identify their combination with an atomic cut as the relevant trigger family. That does not mean either feature is universally broken on its own. It means their interaction exposed a failure in how the engine handled scope restoration across its internal control flow.

This distinction helps keep the compatibility claim precise. The fix is about preserving localizations for their documented duration. It does not promise that every embedded code block runs a fixed number of times. The regex reference separately discusses how optimizations and matching paths affect embedded-code execution frequency.

My preference is to keep stateful regex code limited enough that its intended lifetime is explicit. That is a judgement about readability, while the interpreter still owes correct behavior to programs using the documented constructs. A complicated reproducer can reveal a runtime bug even when a simpler application design is available.

The same development release repairs another regex execution issue. pregexec() again handles zero-length strings, correcting an assertion failure affecting Package-Stash-XS that the notes trace to 5.43.4. That is a separate regression with its own trigger, so it should not be presented as another symptom of the scope-unwinding defect.

December's scope correction restores the relationship between match progress and temporary state. A localization that belongs to a continuing match must survive an internal subpattern or atomic-group transition until the documented restoration point. Perl 5.43.6 repairs the premature unwind while preserving those matching and localization rules.