Perl separates shared string buffers before packing native pointers

Copy-on-write depends on the interpreter knowing when a value changes. A native pointer into a shared string buffer creates a way to write without going through the ordinary Perl operation that would separate the values. Perl 5.41.4 repairs that interaction for the p and P formats in pack, preventing writes through a generated pointer from changing other scalars that share the buffer.

The release history records the snapshot on 20 September. Its release notes describe the fix in implementation terms. The formats now apply SvPV_force to the supplied scalar unless it is read-only, removing copy-on-write from that scalar before supplying the pointer.

I usually choose Perl for text work at a much higher level than native pointers. That makes this fix interesting as an explanation of where the language's ordinary value behaviour depends on implementation cooperation. A low-level escape can affect assumptions made by code that never explicitly asked to share a buffer.

Packing a pointer does not pack the string contents

The function reference for this snapshot defines p as a pointer to a null-terminated string. P packs a pointer to a structure represented as a fixed-length string. These formats differ from a template that places the string's contents directly into the packed result.

That distinction is central to the failure. The result identifies memory containing the value. Code using the pointer can operate on that memory directly. The packed result is therefore connected to the lifetime and state of another object, not merely to the bytes contained in the result itself.

The documentation explicitly assigns a lifetime responsibility to callers of p. The string must not be a temporary value that can disappear before the packed pointer is used. It also documents a null pointer for an undefined input value. These rules show that pointer packing has always required more care than ordinary value encoding.

There is no need to write through a pointer to demonstrate the lifetime distinction. Even a pointer intended only for reading must continue to identify valid storage. Separating a shared buffer addresses a different property, namely whether a write can affect another scalar. Both properties matter at the native boundary, and satisfying one does not imply the other.

Shared storage must separate before external mutation

The release notes identify the problem as other scalars sharing the same copy-on-write string buffer. Logically separate values can reuse storage until an operation requires independent contents. That optimisation relies on the mutation path taking account of the sharing relationship.

A write through an exported pointer does not ask the interpreter to assign a new value. If the pointer still identifies shared storage, changing its bytes also changes what another scalar sees in that storage. That violates the expectation that a modification to one value leaves an independently held value unchanged.

The fix moves the separation earlier. Before returning the pointer for a writable scalar, pack forces the buffer handling that removes copy-on-write. The interpreter can perform that step while it still controls the operation creating the pointer. Waiting for a later native write would be too late, because that write does not necessarily call back into Perl.

This is a specific location for the correction. It does not remove copy-on-write from the interpreter as a whole. It changes the preparation of the value supplied to these pointer formats. Ordinary text-processing code can retain shared-buffer optimisation until another operation requires separation.

Magical scalars remain a separate problem

The release notes explicitly say the change does not make pointer packing safe. For a magical scalar, writes to its buffer will likely disappear on the next read. The notes therefore distinguish buffer isolation from the broader semantics of a scalar whose behaviour involves interpreter-managed operations.

That limit follows the same general boundary. Native code can alter bytes without invoking the operations that a Perl-level assignment would perform. Fixing interference between shared buffers does not make a raw byte write equivalent to an assignment in every kind of scalar. An extension that relies on that equivalence still has an assumption to examine.

It would be misleading to describe the fix as making pack a general-purpose safe mutable foreign-memory interface. The documented improvement concerns one sharing problem. Pointer lifetime, read-only input and magical values retain their own constraints. The snapshot's precise wording is useful because it preserves those distinctions.

I prefer this level of specificity in a memory-related release note. It identifies what now stays independent and names a remaining semantic exception. That is more useful to an extension author than a broad claim that pointer handling has been made safe.

The snapshot also catches a different implicit assumption

Another change adds a warning for possible precedence problems involving !. The notes give examples where a programmer appears to negate a comparison but actually compares the negated first operand. In particular, !$x eq $y groups as (!$x) eq $y.

The suggested alternatives include a negated comparison operator or explicit parentheses around the comparison. This is a source-level assumption rather than a buffer-sharing assumption, but the diagnostic has a similarly narrow purpose. It points out a specific interpretation that may differ from the author's intent.

For the pointer fix, the interpretation occurs at the interface between Perl storage and native memory access. Separating the buffer before exposing it ensures that the exported address no longer refers to another scalar's copy-on-write storage. The remaining restrictions still follow from what the pointer operation actually provides.