Perl 5.31.9 tests Unicode data and property lookup together

A Unicode update changes the data behind a regular expression even when its source text stays the same. Perl 5.31.9 makes that relationship explicit by adopting the Unicode 13 beta and participating in its soak cycle. The same snapshot exposes additional Unicode properties and allows character names inside property expressions. These changes put more character information within reach of the regex engine, while making it necessary to distinguish a character's name from a policy about accepting it.

The 5.31.9 release notes describe the beta adoption as a deliberate testing decision. They say the consortium has approved the existing changes and that relatively few changes affect existing characters. That is a claim about the expected scope of compatibility risk, not a promise that every application sees identical matching results. The release's history file dates this development snapshot to February 20.

Security properties supply data for a policy

The new Identifier_Status and Identifier_Type properties address examination of input for security purposes. The release notes distinguish these properties from the strict Unicode Character Database and connect their addition to work on Unicode regular expression requirements. Perl is therefore expanding the information available to a pattern beyond the familiar character categories. The property mechanism remains the interface through which a program asks about that information.

An identifier policy still needs a definition of what an application accepts. Exposing a classification does not decide whether a service permits a particular alphabet, whether two names are equivalent, or how a rejected identifier should appear to its user. The relevant improvement is that a Perl program can consult these classifications through regex properties. Treating their presence as a complete security system would assign them responsibilities the release notes do not claim.

I value this direction because Perl's built-in matching syntax is one reason I keep choosing it for text work. Access to the relevant character data within that syntax reduces the need for an unrelated lookup mechanism. The benefit depends on keeping the meaning of a property test explicit, particularly when the surrounding program uses the result to accept or reject input.

Character names become property values

The snapshot also accepts \p{Name=...} and its abbreviated \p{Na=...} form. These provide an alternative to \N{...} inside a regular expression, but the two forms are not interchangeable in every respect. The Unicode documentation shipped with the snapshot compares their behavior directly. Its example makes the practical difference visible:

my $which = 'A';
my $pattern = qr/\p{na=latin capital letter $which}/;

The property form permits interpolation of the variable within the name. The documentation contrasts this with the need for evaluation to achieve interpolation using the other form. A program that constructs character-name patterns can therefore use the property syntax without building an evaluated Perl expression merely to select the name. This is a specific change in pattern construction, independent of the addition of Unicode 13 data.

Name parsing also differs. The property form follows Unicode loose matching rules, while the comparison describes the other form as exact. Case and the treatment of spaces, underscores, and hyphens consequently matter differently. A pattern author cannot assume that two spellings receive the same interpretation merely because each appears to refer to a character name. The documentation also identifies outlier names, so a blanket statement that punctuation never matters would be too broad.

Custom names remain another distinction. The existing name escape supports custom aliases and overrides, while the property form does not support custom names. Both forms support name aliases in the comparison. Named sequences, however, are not yet supported through the property form in this snapshot. That limitation matters because a sequence is not interchangeable with a single named character merely because Unicode assigns it a name.

More precise compilation rules accompany the additions

The release also restricts certain matching features while compiling wildcard subpatterns for Unicode property values. The notes explain that those features are inappropriate or interfere with the algorithm used for this operation. Property-value matching has its own compilation requirements. It cannot simply inherit every capability of an ordinary subject-matching expression without regard to the different task.

Extended bracketed character classes receive a related correctness change. The notes say the previous implementation used heuristics when one previously compiled class was interpolated into another. A more reliable method replaces those heuristics. Some constructs that previously compiled may now fail, although the notes identify no known affected examples. This is a compatibility consequence of rejecting erroneous constructs more accurately, separate from the expected small impact of the Unicode data update.

My preference is for a pattern's accepted syntax to correspond closely to its defined meaning. A compiler accepting an erroneous expression is poor protection for an existing program if the resulting behavior was never what its author intended. In this snapshot, broader Unicode access arrives with tighter rules for particular forms of pattern composition. Module authors can exercise the beta data and the new syntax now, while keeping character-name lookup, identifier classification, and the behavior of interpolated character classes distinct in their test cases.