Perl makes wide strings illegal in vec

A string can represent text or a sequence of bytes, but a bit operation needs a definite interpretation of its storage. Perl 5.31.1 now rejects vec on strings containing code points above 0xFF. Its release notes explain that these strings use UTF-8 internally and can produce unexpected results in a bit-oriented operation. The June 20 development release turns a deprecation introduced in 5.28.0 into a prohibition.

The practical question is which positions the operation addresses. A character's ordinal value and the bytes of its UTF-8 encoding are different things. Letting a caller slide between those interpretations makes an offset depend on a representation detail. Rejecting wide characters forces that distinction to be resolved before vec performs its work.

I use Perl for concise text transformations, but that preference does not make every string operation a text operation. This restriction is useful because it gives a specific failure to a program that has reached a bit-level interface with character data that interface cannot represent directly.

Elements have an explicit bit width

The vec documentation shipped with this version defines the operation in terms of elements in a bit vector. Its BITS argument gives each element's width, and OFFSET selects an element. The width must be a power of two from one through 32, with 64 available on platforms that support it.

When the width is eight, elements coincide with bytes. Larger widths group bytes into big-endian values, using the correspondence documented for pack and unpack formats. Widths below eight divide a byte into smaller groups with their own documented bit numbering. None of these definitions gives vec a character-decoding role.

A small example uses only the byte-sized case.

my $buffer = '';
vec($buffer, 0, 8) = 0x50;
print vec($buffer, 0, 8);  # 80

The assignment writes a byte-sized element. Reading it returns an unsigned integer. The example follows the documented ability to assign through vec and the manual's byte value for 0x50. It does not depend on interpreting a multi-byte character encoding.

Writing beyond the current end of a string extends it with enough zero bytes. Reading an element outside the string returns zero. A negative offset is invalid for a write. These rules make sense for a bit vector with explicit element positions. Treating it as a sequence of abstract Unicode characters would require a different set of definitions.

The UTF-8 flag is not the prohibition

The manual describes an important qualification. If a string has the internal UTF8 flag set, vec attempts to convert it to a one-byte-per-character representation. A fatal error occurs if it contains characters with values of 256 or higher. The rule therefore concerns the string's code points, rather than rejecting every scalar that happens to have used an internal UTF-8 representation.

That distinction matters when diagnosing an error. A string whose characters all fit into byte values can still be representable for the operation. A string containing a wide character cannot be converted in that way. The restriction is about whether the intended element data fits, not merely a label attached to the scalar.

The release notes describe this as enforcement of an existing deprecation. Programs that already received the warning in 5.28.0 now encounter a fatal condition in this development release. Describing it as a wholly new interpretation of vec would miss that transition. The intended byte-oriented use remains available, including ordinary bit vectors.

A program that actually needs to manipulate encoded text as bytes has a different requirement from one that needs character ordinals. That decision belongs before the call. Reinterpreting the internal representation silently cannot establish which requirement the caller intended, especially when the result is later treated as text again.

String bitwise operators follow the same boundary

Perl 5.31.1 also prohibits the remaining uses of code points above 0xFF in string bitwise operators. The release notes say some uses were already illegal after an earlier deprecation cycle. The remaining cases had been deprecated in 5.28.0 and are now prohibited as well.

The vec manual notes that strings created with vec can be manipulated by logical bitwise operators. That relationship explains why both changes appear together. A byte-vector operation should not accept wide character data merely because a different syntactic route reaches it. The data model needs to remain consistent across operations that treat strings as collections of bits.

My preference is for a clear boundary at that point. Perl's convenient reuse of scalars does not remove the need to decide whether a value is a character string or binary data. A specific rejection is easier to interpret than a result whose meaning depends on the scalar's incidental storage format.

Other finalization fixes remain separate

The release also repairs closing a pipe when a signal handler calls die or exit while close waits for the child. Perl now clears the PerlIO object from the IO scalar before waiting, preventing a second attempt to close an already closed object. That is a resource-lifetime correction, independent of the vec restriction.

Another correction updates Storable's handling of regex objects during retrieval. The notes say those objects were not counted correctly for object identifiers, which could corrupt the reconstructed structure or produce a runtime error. Here too, the issue lies in a representation contract, but it concerns serialized object relationships rather than byte-sized elements.

These changes define the scope of this development delta. The vec restriction does not claim to redesign Perl's entire string system. It enforces a particular limit at a bit-oriented operation, while still allowing strings whose characters can be represented as single byte values. That is enough to remove the documented ambiguity for code points above 255.