Indented heredocs in Perl 5.25.7

A multiline string inside an indented block has two competing layouts. The program's indentation describes its control flow, while the string's indentation may belong to the data. Perl's new indented heredoc lets the programmer describe both without a separate substitution. In the Perl 5.25.7 release notes, the ~ modifier allows whitespace before the closing delimiter and removes that same whitespace from each content line. The release's history file dates it to November 20, 2016.

This is a development release, so the new spelling belongs to the interpreter being developed. It is useful to examine the rule on its own terms without assuming that an existing Perl installation accepts it. I value Perl for short text transformations, and a string construct that preserves the intended output without an extra regular expression fits that preference. The interesting part is how narrowly the parser defines the whitespace it removes.

The delimiter defines the prefix

The operator documentation shipped with 5.25.7 describes an ordinary heredoc as a line-oriented quoting form. An opening operator names a terminating string. Subsequent lines supply the value until the parser encounters that terminator. With an ordinary heredoc, the closing string must occupy its line without surrounding whitespace. That forces a choice when the expression sits inside a block. Either the text and terminator move to the left edge, or the program needs another operation to remove unwanted indentation from the value.

The new modifier makes the closing line the reference for indentation. This example follows the form documented in the release notes, with one additional level of indentation inside the value:

if (1) {
    print <<~'REPORT';
        status
          ready
        REPORT
}

Eight spaces precede REPORT on the closing line. Perl removes eight spaces from the beginning of each content line, leaving two spaces before ready. The resulting string still has a newline after each content line. The indentation of print does not determine the removal amount. Moving the statement without changing the closing delimiter therefore does not redefine the string's baseline.

That distinction matters for embedded text whose internal layout carries meaning. A line that deliberately starts farther to the right retains the additional whitespace. The parser does not flatten every line to the left edge, and it does not infer a baseline by inspecting the least-indented content line. The closing delimiter states the prefix explicitly. Someone reading the source can determine the output indentation from that line and the content that precedes it.

Whitespace must match exactly

The versioned operator documentation requires the same starting whitespace on every content line, except a line containing only a newline. A line that lacks the required prefix causes an error. Tabs and spaces may appear together in that prefix, but the parser matches their actual characters. A tab does not stand for eight spaces merely because an editor displays it at that width.

This means two source lines that appear aligned in an editor can behave differently. A closing delimiter preceded by spaces cannot remove a visually equivalent tab from a content line. The rule follows the bytes used for indentation, without assigning meaning to the editor's tab stops. Consistent indentation is therefore part of the string's syntax when this modifier is present.

I prefer this explicit failure to an automatic guess about mixed whitespace. Removing a different prefix could change generated text while leaving the program apparently successful. Here, the delimiter describes a requirement, and a content line that violates it prevents Perl from accepting the string. Extra indentation after that required prefix remains ordinary data.

Blank lines deserve the exception the documentation gives them. A line containing only a newline need not carry the indentation prefix. That permits an empty line within a paragraph or configuration fragment without padding the source with spaces. It does not relax the matching rule for a line that contains text.

Quoting remains a separate decision

The modifier works with the existing heredoc quoting forms. A single-quoted delimiter makes the content literal, while a double-quoted delimiter applies double-quoted string interpolation. An unquoted identifier also uses double-quoted behavior. Adding ~ changes the indentation handling without choosing an interpolation policy for the programmer.

my $state = 'ready';
if (1) {
    print <<~"REPORT";
        state=$state
        REPORT
}

This prints state=ready followed by a newline. Replacing the opening delimiter with <<~'REPORT' would preserve the literal text $state. The closing line remains the unquoted word REPORT in either case. The documentation also permits whitespace between the modifier and a quoted delimiter, so the quoting choice does not depend on pressing the quote directly against ~.

Backtick delimiters remain available too. The operator documentation says that form interpolates the content and executes it through the shell, returning the command output. Indentation removal does not turn that construct into a literal string. For examples intended to represent data, single quotes make the absence of interpolation explicit and avoid introducing command execution into the explanation.

Replacing a substitution changes the contract

The same versioned operator manual documents the older approach of assigning a heredoc and removing leading whitespace with s/^\s+//gm. That expression asks a different question. It removes matching whitespace at the start of lines, without distinguishing indentation needed by the source from additional indentation belonging to the value. The new modifier permits the programmer to retain that distinction in the heredoc itself.

For text with deliberately nested lines, replacing a broad whitespace substitution with an indented heredoc can therefore change the output. The extra spaces before ready in the first example survive the new rule. That is useful when the source layout should preserve relative indentation, but it also explains why the two spellings are not mechanically interchangeable.

My interest in this change is the ability to express a particular text transformation directly in the quoting syntax. The programmer supplies a baseline through the closing delimiter, chooses interpolation through the opening quotes, and leaves additional content indentation intact. Those decisions remain visible beside the text that they affect. The resulting value retains its final newline unless the program separately removes it, just as the operator documentation describes for heredocs generally.