January 20, 2024·4 min read·engineeringperlxslocales

Perl 5.39.7 improves the boundary between XS and locale data

The January development snapshot repairs XS stack compatibility and gives native extensions a safer interface to locale strings.

A locale query looks simple until a native extension has to decide what its returned bytes mean. Perl 5.39.7 adds an interface that returns a Perl scalar for locale information, so extension authors no longer have to manage the UTF-8 status of a returned C string themselves. The same snapshot adjusts the types used by XS argument macros to restore compatibility for default interpreter builds. Both changes concern the boundary where Perl programs depend on native code, even when their own source contains no C.

The release history records Max Maischein's release of 5.39.7 on 20 January. The release-tagged notes describe changes since 5.39.6. This is a development snapshot on the way to 5.40, so the useful question is what module authors can learn from it before the stable interpreter arrives.

I still value Perl for short text transformations. Regular expressions are part of its grammar, and that makes a small program an effective way to express a single restructuring job. Changes underneath those programs matter because the compact expression at the command line can still depend on platform conventions and native modules.

Returning locale information as a scalar

The new XS function is sv_langinfo. According to the release notes, it supplies the information available through Perl_langinfo8, but returns an SV instead of a char *. The notes explicitly identify it as the preferred XS interface to the C library's nl_langinfo function. Its purpose is to remove the caller's responsibility for the UTF-8 status of the result.

That return type changes where the responsibility lies. A pointer to characters gives the caller access to a sequence of bytes. The extension still needs to represent those bytes appropriately when it passes the result into Perl. Returning the interpreter's scalar representation lets the interface carry that responsibility across the native boundary. This does not require a new Perl-level locale API. The notes say that I18N::Langinfo continues to provide the information to Perl programs.

The module documentation shipped with this snapshot explains another relevant detail. langinfo queries the current underlying locale whether or not the call appears inside the lexical scope of use locale. That distinction matters when reading code. Absence of the pragma does not make the queried information independent of the process's locale.

A small example can ask for the current character set and decimal separator without assuming a particular answer:

use strict;
use warnings;
use I18N::Langinfo qw(langinfo CODESET RADIXCHAR);

my $codeset = langinfo(CODESET);
my $radix = langinfo(RADIXCHAR);
print "Character set: $codeset\n";
print "Decimal separator: $radix\n";

These are existing Perl-level queries, not new syntax in 5.39.7. They make the distinction concrete. The caller requests named properties of the current locale, while the implementation handles their representation. The new native interface improves the corresponding operation for XS authors.

Linux supplies additional locale categories

I18N::Langinfo moves from 0.22 to 0.23 in this release. Its release-note entry says that it now handles Linux's locale categories beyond those defined by POSIX. The module documentation lists information about addresses and telephone formatting, as well as paper dimensions and measurement conventions.

These entries expose operating-system locale data. They do not establish universal values that every platform supplies. The documentation says the Linux-specific items return stub values on other platform types. For paper dimensions, for example, the documented Linux values describe sheet sizes in millimetres. A program that queries them needs to distinguish the availability of a constant from the availability of useful platform data.

The distinction is especially relevant to portable modules. A function can exist on every supported platform while some of its answers remain platform dependent. The same documentation describes fallback behaviour on systems without nl_langinfo, including limits on which values can be reconstructed. The expanded Linux support therefore increases the information available on that platform without removing the need to understand the other implementations.

I prefer this kind of addition when it keeps the operating-system dependency explicit. A locale interface should expose the information the system supplies and document its limits. Treating every returned value as equally authoritative across platforms would conceal a difference that application code may need to handle.

XS argument types return to compatibility

The second internal change affects dAX, dITEMS and consequently dXSARGS. They now declare AX and items as Stack_off_t instead of SSize_t. The release notes describe this as restoring compatibility with the situation before 64-bit stack support for default builds, where Stack_off_t is I32.

This is an extension-facing change. An application written entirely in Perl does not name those macros, but an XS module can depend on them. The important detail is the use of the stack-offset type selected by the interpreter. The notes do not claim that every build now uses a 32-bit stack offset. They identify the default-build case and explain why choosing Stack_off_t restores its previous compatibility.

There is no new user-level spelling to adopt here. Module authors have a concrete declaration change to inspect when compiling against the snapshot. That makes the development release useful even if its list of language changes is empty. Compatibility work at the C boundary needs feedback from the extensions that use that boundary.

The release also updates several bundled modules, including Exporter and Math::BigInt. Their presence is a reminder that evaluating a Perl snapshot involves the distributed modules as well as interpreter code. For this release, the most specific changes concern how native code receives stack offsets and how locale strings enter Perl. Both give extension maintainers a defined piece of behaviour to examine before the next stable release.