Perl applies UTF-8 argument handling to variables created by -s

Two identical command-line arguments could become different Perl strings depending on whether one passed through the interpreter's switch parser. With -CA, positional arguments received the requested UTF-8 treatment, while values assigned to globals by -s did not. Perl 5.43.4 fixes that inconsistency in the development interpreter.

The release notes illustrate the defect using the character U+00E9, LATIN SMALL LETTER E WITH ACUTE. Its positional argument became one code point. The switch value retained two UTF-8 bytes as separate character values. The history records date this release to October 22.

Perl remains my choice for short text transformations, so command-line representation is part of its usefulness to me. An option value and a positional value containing the same text should not acquire different meanings because of an internal parsing path. The new behavior makes the selected argument policy apply consistently to both.

Switch parsing removes an argument before the program sees it

The versioned invocation documentation describes -s as rudimentary switch parsing. A switch after the program name is removed from @ARGV and sets a corresponding variable in the main package. A switch written as -xyz=abc assigns the value abc to that variable.

That removal explains how a text-conversion gap can depend on argument position. A positional argument remains available through @ARGV. A parsed switch value has already taken another route into a global. Applying conversion only to the remaining positional values misses the values extracted earlier by switch handling.

The release-note example compares $foo and $ARGV[0] with printf's %vx format. Before the fix, the same visible accented character produced c3.a9 through the switch and e9 through the positional argument. That representation makes the difference visible without relying on a terminal to render the two strings differently.

A diagnostic in the form used by the release notes is useful for understanding the correction. It assumes the shell supplies the visible arguments in UTF-8.

perl -CA -s -e 'printf "%vx\n", $_ for $foo, $ARGV[0]' -- -foo=é é

In the corrected interpreter, both values contain chr(0xE9), so each diagnostic line shows e9. This is a comparison of internal character interpretation. It avoids treating visually identical output as proof that the strings have identical contents.

The selected flag has a narrow purpose

The A part of -C selects argument handling. The invocation reference lists separate settings for standard input and output, along with default layers for other streams. Selecting -CA therefore does not mean every input or output operation in the program has received the same policy.

The equivalent environment setting is PERL_UNICODE=A, which the release notes explicitly include. That matters because the behavior can be selected outside the script's source and outside its visible command invocation in a wrapper. The correction applies to the policy in effect, rather than requiring one particular way of spelling the setting.

The reference also gives a qualification that should accompany any explanation of these flags. They assume the internal upgraded byte encoding rather than validating strict UTF-8 input. The new fix extends consistent handling to switch values. It does not turn -C into an input-validation facility.

Those are separate correctness questions. Consistency asks whether identical arguments take the same interpretation through both routes. Validation asks whether the input representation satisfies an encoding's rules. A fix for the first does not establish the second, even though both involve UTF-8-related processing.

Character behavior depends on the decoded value

Keeping an encoded byte sequence as separate character values changes the text available to later operations. A length calculation or a comparison acts on the resulting string, not on what the shell command looked like. The bug can therefore remain hidden when output looks plausible while an operation depends on the characters actually stored.

The release-note diagnostic isolates that problem before introducing application logic. It compares the values immediately after argument processing. That establishes which interpreter path differs, without attributing the discrepancy to a later regex, a file's encoding layer, or terminal display. A caller comparing the parsed switch to the positional value can then reason about their equality as text. The repair removes the representation mismatch before either value reaches the program's own processing rules.

I find that style of reproducer particularly useful for Perl text issues. Showing code-point values identifies the representation error directly. It also keeps the repair's scope clear when the same release contains other Unicode-related fixes.

The invocation documentation discourages -s because of its limitations, including awkward variable names and warnings, and points to Getopt::Long for more flexible parsing. Existing scripts still need the documented interpreter switch to behave consistently. Perl 5.43.4 now applies the requested -CA handling to the globals that switch creates, matching the positional argument path.