March 20, 2018·4 min read·engineeringperldiagnosticscompatibility

Perl 5.27.10 repairs failures in diagnostic code

Perl 5.27.10 fixes regex and CPAN regressions while making Carp less likely to fail during the error reporting that exposes them.

Error reporting can invoke enough of a language runtime to cause a second failure. Perl 5.27.10 contains an instructive example in Carp, whose stack traces need to inspect arguments and represent objects while the program may already be in trouble. The release notes describe changes that avoid recursion through overloaded objects and work around problems in the interpreter stack. These sit alongside fixes for a regular-expression assertion failure and a regression affecting Crypt::Perl.

The tagged release history records March 20 for this development release. Its interest is in the conditions under which compatibility and diagnostics fail. Some bugs occur during ordinary execution. Others require an error handler to examine state that normal execution does not inspect, or require a native extension to create an object in a way pure Perl code would not.

I retain a practical affection for Perl as a text tool. That convenience makes dependable diagnostics valuable, since a compact expression can still involve substantial interpreter machinery. A release that repairs the reporting path helps preserve the ability to understand failures in that machinery.

Object formatting can call back into the reporter

Carp 1.49 no longer assumes that an object cannot have overloaded behaviour unless the overload module has loaded. The release notes identify XS-created objects as a counterexample. Previously, an XS-defined overload method that itself called Carp could produce infinite recursion. The reporting code could enter the same behaviour it was trying to describe.

The Carp implementation at this release explains why reference formatting needs special care. It wants an ordinary representation of a reference without invoking overloaded stringification. Invoking a method merely to display an argument can execute application code at an inconvenient time. If that code invokes the reporter again, the attempt to describe an error becomes recursive.

The implementation uses a private _StrVal function for this purpose. Where the overloading module is available, it builds a function that disables overloading while stringifying its argument. The comments describe additional compatibility paths for older interpreter and module combinations. The aim is consistent reference formatting without depending on the potentially failing application-defined conversion.

That behaviour is useful precisely because an object may have valid overloaded behaviour during normal execution. A diagnostic routine does not need to reject the object or redefine its meaning. It needs a representation that does not call the same conversion code again. Separating those operations makes the reporting path less dependent on the code being reported.

Loading dependencies after a syntax error

The release notes also explain why Carp avoids overload::StrVal. Some older versions of overload load Scalar::Util at runtime. That loading can fail when Carp has been invoked after a syntax error. The module cannot treat the error-reporting environment as equivalent to an ordinary point in a running program.

Carp has to be available to die handlers and to code invoked after parsing has already failed. Its source comments explicitly prohibit loading modules on demand for the reference-formatting operation. Where suitable support exists, it prepares its helper during module initialization. The implementation therefore considers when a dependency loads as well as which function it supplies.

This is a concrete reason that replacing a local helper with a familiar library call can change correctness. The library function may produce the right result in an ordinary call while making an unsuitable demand on a damaged execution context. Carp needs reference formatting under narrower conditions than application code usually assumes.

A separate change concerns argument inspection during stack-trace generation. The release notes identify longstanding problems associated with the non-reference-counted stack. In the source, Carp copies arguments under eval and substitutes an explicit unavailable-argument message if that operation fails. The comments explain that inspecting arguments can expose stack problems even where ordinary program flow would avoid touching them.

The fallback loses part of the original argument description, but allows the diagnostic to retain the information it can still obtain. This is a defensible compromise for a reporter. A partial trace with an unavailable argument is more useful than losing the report to a failure caused by inspecting that argument.

Repairing the surrounding development release

The same release lists the Crypt::Perl compatibility failure as issue 132900 and the regex assertion failure as issue 132163. Its selected fixes also include corrected line numbers in multiline substitutions. The notes do not supply enough detail to infer the precise input behind either numbered failure. Their inclusion establishes that the regressions were addressed, without supporting a broader claim about every use of cryptography or regular expressions.

Another diagnostic change narrows an earlier restriction on goto. Perl 5.27.8 had prohibited entering the argument of a binary or list expression to prevent stack corruption. In 5.27.10, entry into the first argument of a fixed-arity operator is allowed again because that case does not cause the corruption. The distinction removes an unnecessary rejection while preserving the restriction on unsafe cases.

I prefer this kind of explanation to a general assurance of compatibility. It identifies the execution condition that matters and narrows the rule accordingly. Carp applies a similar discipline to errors, preserving what it can report while avoiding operations that can make reporting fail. The source records the constraints behind those choices, including the interpreter stack and the timing of dependency loading.