Perl 5.25.10 repairs indirect method parsing

Deciding whether an expression is a method call can itself invalidate the memory that the parser is reading. Perl 5.25.10 fixes a case where checking for an indirect object method call reallocates the line buffer, after which parsing continues through pointers into the old buffer. The release notes identify that failure as issue 129190. A separate correction handles a heap use after free or buffer overflow when the deprecated $# variable appears as the object in an indirect call. The history shipped with the release records February 20, 2017.

These are development-release fixes with consequences below the level of ordinary syntax errors. A parser may reject an unusual expression, but it still has to retain valid references while deciding what the expression means. An obsolete variable combined with an uncommon calling form does not remove that requirement. The two reports describe distinct routes through the same area of the interpreter, and neither needs a successful method invocation to explain the memory problem.

I value Perl's compact treatment of text, especially when the program is short. That convenience makes parser correctness a practical concern. The cost of supporting an expressive grammar includes handling combinations that few people would choose to put into new code.

Buffer lifetime during recognition

The line-buffer correction concerns ownership across an operation that can allocate memory. Reallocation can move the buffer to a different address. A pointer retained from before that move no longer identifies the current buffer, even if its offset once made sense. Continuing through the retained pointer can therefore read storage that the parser no longer owns in the same way.

The release notes describe this happening while checking for an indirect object method call. That detail locates the failure during recognition, before any argument about what a method should do at runtime. A language implementation must preserve its memory invariants while considering a possible interpretation of the input. Rejecting the eventual expression would not retrospectively make the intermediate memory access valid.

Issue 129274 involves the deprecated $# variable as the indirect object. The notes name both heap use after free and buffer overflow as possible outcomes. They do not give an exploitability assessment or describe a generally usable attack. Calling the release a fix for these parser memory errors is justified. Claiming that it closes a demonstrated remote-code-execution vulnerability would go beyond the information supplied.

Regex execution also depends on scope

Several other fixes concern the interpreter's bookkeeping during regular-expression execution. Since 5.24.0, an obscure combination of regex code blocks from multiple sources could use the wrong current pad. The release mentions embedded qr// objects as one way the combination could arise. The resulting behavior could include a crash or incorrect results.

That failure matters because combining regex components also combines their associated execution context. The written pattern alone does not describe all the state needed to execute embedded code. The fix addresses the interpreter's selection of that state when pieces originate in different places. It is separate from changing what an ordinary character class matches.

Another correction restores local() changes when matching backtracks over a code block inside a pattern. Backtracking revisits an earlier matching choice, so the corresponding temporary changes must be undone at the appropriate point. Leaving those changes in place makes an unsuccessful matching route affect later execution. The notes identify this independently as issue 126697.

For my use of Perl, these fixes reinforce a preference for keeping one-off text programs direct. Code inside a pattern can make the execution model harder to follow because matching decisions interact with language state. That is a judgement about readability, not a claim that every regex in a script exercises these bugs.

A compiler update has compatibility costs

The same snapshot adds support for building Perl on Windows with Visual Studio 2015 and its Visual C++ 14.0 compiler. Its rewritten C runtime makes the existing fix for two socket close() bugs unworkable in its current form. The release therefore effectively reverts that fix for VS2015 and later compiler builds, prioritizing the ability to build with that toolchain.

This is an unusually explicit account of a compatibility tradeoff. A new compiler can require a change to runtime integration even when the Perl program has no new syntax. The notes also distinguish unaffected configurations. GCC builds and Visual Studio versions through VS2013 retain the earlier socket fix unchanged.

Mixing an interpreter built with one of those older toolchains with XS modules built using VS2015 can create compatibility problems. The reverse combination has the same warning. Some trouble may follow from the different socket-fix behavior, while the rewritten C runtime can introduce incompatibilities of its own. Consequently, the compiler used to produce an XS module remains relevant when identifying the affected installation.

More precise inspection of interpreter operations

The internal op_class() API added in this release gives extension and interpreter developers a more precise answer about an operation's allocation. An existing OP_CLASS() macro can report OA_BASEOP_OR_UNOP, meaning that operations of that type are usually allocated using either structure. The function can instead return OPclass_BASEOP or OPclass_UNOP for the particular allocation.

That distinction fits the release's broader concern with concrete interpreter state. Parser pointers need to identify live buffers, regex code needs the correct pad, and internal inspection needs the actual allocated structure. The new API makes the last distinction explicit for callers that need it. Those callers can use information about the particular operation instead of treating a classification of its type as a complete description of its storage.