Unicode 15 changes what Perl character classes recognize

A regular expression can accept new input after an interpreter upgrade even when the pattern has not changed. That happens when the pattern asks about a Unicode character property and the interpreter adopts a newer character database. Perl 5.37.5 adds support for Unicode 15.0, according to its release notes. The release history records the development snapshot on October 20.

This is relevant to Perl because character classes are part of the ordinary pattern language. An expression such as \d refers to a defined set of characters under the selected matching rules. Updating that definition can expand the recognized set without adding any new punctuation to the language.

New characters include new decimal digits

The Unicode 15.0 release overview dates the standard to September 13. It adds 4,489 characters, bringing the total to 149,186. The additions include the Kawi and Nag Mundari scripts, as well as new CJK ideographs and symbols. This gives the Perl update a concrete data change to incorporate.

The numeric additions are especially instructive. Unicode's migration notes identify new sets of decimal digits for Kawi and Nag Mundari. The versioned character database classifies U+1E4F0 as NAG MUNDARI DIGIT ZERO with general category Nd. It is a decimal digit even though its code point is far outside the ASCII digit range.

The character-class manual shipped with Perl 5.37.5 explains the relevant distinction. Under Unicode matching, \d corresponds to the decimal-number category. Under the /a modifier, it matches the ASCII range [0-9] instead.

use strict;
use warnings;

my $digit = "\x{1E4F0}";

print "Unicode decimal digit\n" if $digit =~ /\A\d\z/u;
print "ASCII decimal digit\n" if $digit =~ /\A\d\z/a;

In this release the first condition recognizes the new digit. The ASCII condition does not. The code-point escape avoids depending on a font's ability to display the character in the source editor, while the explicit modifiers expose the two policies being compared.

I value Perl's regular-expression integration because it makes text transformations concise. That also makes the definitions behind short character classes worth reading. A one-character predicate in a pattern can represent a much larger policy than its spelling suggests.

A digit class is an application decision

Neither matching mode is universally the right one. A field specified as ASCII digits has a different contract from a field intended to accept decimal digits across writing systems. The choice belongs to the format or application, and the expression should state that choice accurately.

The Perl manual also distinguishes decimal digits from other numeric characters. Numeric_Type=Digit is a separate property whose contents differ from the decimal-number category used by \d. Its examples include circled and subscript digits. Treating every character with some numeric meaning as interchangeable with an ordinary decimal digit would erase a distinction the Unicode properties preserve.

The update therefore concerns classification as well as repertoire size. The presence of a character in a font or a string does not establish which property it has. A property-based expression delegates that question to the interpreter's Unicode data. An explicit ASCII range makes a narrower statement whose membership is visible in the pattern itself.

For a text-processing tool, that distinction affects what an unchanged filter includes. For an input validator, it affects the language of accepted inputs. Both uses can be intentional. Documenting the chosen matching rules is more informative than describing the pattern simply as a check for numbers.

Property names also have stability rules

Unicode 15.0 strengthens its alias stability policy. The release overview says that property aliases and property-value aliases will retain their exact spelling, in addition to the existing guarantee against removal. It describes this as a formal guarantee of an established practice intended to keep property references in regular expressions valid.

That is a different kind of stability from freezing the membership of a property. A property's name can remain valid while newly assigned characters become members. Code using that name can continue to compile and still recognize more text under a newer database. Those outcomes are compatible because spelling stability and data stability are separate promises.

The standard also adds a domain stability policy. An existing property of characters cannot become a property of strings, or the reverse. The distinction helps implementations preserve what kind of object a property describes, even as the supported repertoire grows.

My preference is to treat the Unicode version as part of the explanation for a text tool's behavior. The October snapshot provides a specific version to examine, with new characters and documented properties. For the digit example, the meaningful change is that Unicode-aware matching recognizes a newly assigned decimal character while the explicitly ASCII pattern continues to exclude it.