Perl rejects incomplete hexadecimal and binary literals

An incomplete numeric literal should not quietly become a different expression. Perl 5.31.2 makes a bare hexadecimal or binary prefix a fatal error after the previous development release changed how the parser divided it into tokens. The release notes say that 5.31.1 could treat the leading zero as a number and leave x or b for the following token. That could produce silent changes in behavior.

The July 20 release entry therefore records more than an additional diagnostic. The development sequence exposes a choice about malformed source. Continuing with an alternative tokenization can sometimes construct a program, but that program may not express the intended value. The new failure stops that reinterpretation at the point where the expected digits are missing.

I prefer a direct compiler error for this case. Perl's compact syntax is useful for one-off work, but compactness makes it especially important that an unfinished number not acquire a meaning from the adjacent text. A diagnostic about the literal gives a more useful explanation than an unexpected result from a program the author did not mean to write.

A prefix needs digits

The new diagnostic is No digits found for %s literal. The diagnostics manual in this version classifies it as fatal and defines the two cases. There are no hexadecimal digits following 0x, or no binary digits following 0b. That is the complete stated condition, so it should not be widened into a claim that every malformed number now receives the same message.

The data syntax manual shows ordinary hexadecimal and binary constants. A prefix is part of the notation for a value, followed by the digits that supply that value.

my $hex_value = 0xff;
my $binary_value = 0b011011;

Both examples use complete forms documented for this release. Removing all digits after either prefix does not express a smaller value in that notation. It leaves the notation incomplete. Perl 5.31.2 diagnoses that condition instead of interpreting the zero independently.

This distinction is about Perl source code. The same data manual says that a quoted string containing a hexadecimal, octal or binary representation does not automatically convert to the corresponding integer. Functions such as hex and oct perform the relevant conversions. A source-literal diagnostic therefore should not be confused with a general input-validation rule for every string a program receives.

Why the previous behavior was dangerous

The release notes identify 5.31.1 as the point where parsing incomplete hexadecimal or binary literals changed. The scanner could return the zero as the literal and leave the prefix letter to participate in the next token. Whether the resulting source remained valid depended on what followed. The problem was the possibility of valid code with an unintended meaning, not merely an error whose wording was less clear.

That is a different failure mode from rejecting a literal and reporting its location. A fatal error prevents execution of the incorrectly interpreted source. Successful parsing under a different token split can make the error harder to associate with the original missing digits. The interpreter has accepted a structure that the programmer may never have intended.

Generated source makes this distinction particularly relevant. If a generator emits a prefix but fails to emit its digits, the compiler should expose that incomplete output. Treating a fragment as another expression can hide the generator's defect. This is an implication of the documented tokenization change, not a claim that the release notes identify a particular affected generator.

The narrowness of the fix is useful. It addresses missing digits at these two prefixes. It does not remove the documented ability to put underscores between digits for legibility. The manual continues to show grouped binary forms, including 0b1010_0110. An incomplete literal and a complete literal with permitted separators are different syntax cases.

Compilation also avoids deep recursive walks

The same release changes most functions that recursively walk an operation tree during compilation to use non-recursive traversal. Its notes connect this work to stack overflows on deeply nested trees, giving a long nested conditional expression as an example. They specifically call out automatically generated code as a situation where such trees can appear.

This is another compiler concern with a different mechanism. The missing-digit check prevents an invalid fragment from receiving an unintended interpretation. Non-recursive traversal allows a valid but deeply nested representation to be processed without exhausting the C stack in those walks. One is a source-acceptance decision, while the other concerns the resources needed to compile accepted source.

The notes say the traversal change is particularly noticeable when compilation runs in another thread, because thread stacks tend to be small. That observation explains why the same generated source can expose a problem in one execution arrangement and not another. It does not establish a universal depth limit or a performance number.

My interest in terse Perl programs does not make compiler maintenance irrelevant. Text-processing expressions can be assembled by tools, and their compact written form can still produce a deep operation tree. The release improves the distinction between source that should fail for a clear reason and source that should compile without an avoidable stack failure.

Errors retain their intended meaning

Perl 5.31.2 also changes eval_pv so its croak_on_error behavior throws even when the exception is a false overloaded value. The notes separately add a G_RETHROW option to eval_sv. These repairs concern how an error propagates through interpreter interfaces, rather than the spelling of numeric literals.

They fit the same practical requirement that an error remain recognizable as an error. An incomplete base prefix now stops compilation with a specific diagnostic. A false-valued exception no longer defeats the documented throwing behavior of eval_pv. Each correction makes the interpreter follow an explicit contract at the boundary where a caller needs to distinguish success from failure.