Perl 5.29.2 adopts Unicode 11 with local word-break rules

Updating a character database can change how an existing string behaves even when the string contains no newly assigned characters. Perl 5.29.2 adopts Unicode 11.0, but its release notes discuss two decisions beyond importing new data. Perl retains its own treatment of horizontal whitespace in word boundaries, and it adopts new Georgian casing behaviour despite a documented problem with font availability.

The release history dates this development release to August 20. Its Unicode changes illustrate why a language implementation needs to explain which standard rules it follows directly and which rules it tailors. Saying only that Unicode support has advanced would omit both an intentional compatibility decision and a change visible to readers of Georgian text.

I find Perl useful because matching and transformation are ordinary language expressions. That integration also makes the semantics of character data important. A change in a boundary or case mapping can alter a short transformation without requiring any change to the expression itself.

Whitespace stays grouped

The release notes say Perl keeps sequential runs of horizontal whitespace together for the Word_Break behaviour. Unicode 11 moves closer to that approach, but retains breaks for some whitespace characters, including TAB, NO BREAK SPACE, and FIGURE SPACE at U+2007. Perl explicitly keeps its existing tailoring for those cases.

This decision concerns boundaries, not deletion or replacement of the whitespace itself. A boundary matcher decides where a division can occur. Keeping a run together avoids introducing a division between its members under the tailored rules. It does not make the characters identical in every other operation or remove the difference between their code points.

The regex backslash documentation at this tag distinguishes ordinary \b from \b{wb}. Plain \b finds transitions between word and non-word characters. The braced form implements Unicode word boundaries with Perl tailoring. Its handling of natural-language text can therefore differ from the familiar transition test.

The manual gives apostrophes and decimal punctuation as examples of the difference. A Unicode word boundary can retain an apostrophe inside a word and avoid dividing a decimal number at its point. These are segmentation decisions that a simple test for movement between \w and \W does not express. Choosing the braced form changes the question being asked of the text.

There is also an exception to whitespace grouping when a final horizontal space precedes certain combining or formatting characters. The manual describes boundary property values Extend, Format, and ZWJ in that context. The exception prevents a simplified description of whitespace grouping from replacing the complete documented boundary rules.

That level of detail matters for code that uses boundaries to divide text into pieces. A change in where the matcher reports a boundary changes the resulting pieces even if every character survives unchanged. Perl preserves the stated whitespace tailoring while taking the broader Unicode update, so adoption of the new standard is not synonymous with abandoning all earlier local behaviour.

Georgian gains case mappings

The Unicode 11 version description explains the Georgian change. New Mtavruli capital letters have mappings to existing Mkhedruli letters. The existing letters move from the general category for other letters to the category for lowercase letters. This alters properties and casing for characters that programs could already process before the update.

The standard also notes that Georgian does not use ordinary titlecasing in the same way as other bicameral scripts. Mkhedruli letters do not receive titlecase mappings to Mtavruli capitals. An implementation cannot assume that adding uppercase mappings necessarily implies an identical titlecase transformation. The distinction is part of the script behaviour the new data describes.

Perl release notes report that many devices lack fonts for the new capitals, making transformed text difficult to read. They say delaying that portion of the update has been suggested. The development release nevertheless retains it, with the possibility of reconsideration before the next stable release. This is a decision stated at the time, without a claim about how font support will develop.

A correct character conversion and a legible display are separate requirements. The interpreter can follow the new mapping while a reader sees unsupported glyphs. That does not make the Unicode mapping arbitrary, but it explains why adopting data can have consequences beyond matching and string comparison. Display software and fonts participate in the result the reader receives.

I consider the explicit explanation useful. It identifies why Perl retains the change for development users and which practical problem could cause reconsideration. That is more informative than treating every data update as behaviourally invisible.

Boundary standards can change existing results

The regex documentation warns that Unicode boundary definitions can change between versions, including their treatment of existing characters. It also distinguishes default segmentation from language-specific processing. Japanese and Thai can require dictionary lookup to determine word boundaries accurately. A Unicode boundary construct supplies a defined default, without claiming to solve every linguistic segmentation problem.

Perl 5.29.2 consequently combines a new character standard with a documented local policy. Horizontal whitespace retains the chosen Perl grouping, while Georgian receives new property and casing data. Both decisions can affect text processing, and each needs its own explanation because one preserves existing behaviour while the other changes it.