Go 1.9.3 tightens URL userinfo parsing

A successful URL parse can encourage a stronger assumption than the parser actually supports. If a library returns a username and password, application code may treat them as ordinary credentials and pass them to another component. Go's latest maintenance release closes a specific gap in that assumption. The Go 1.9.3 release record identifies a security correction in net/url, alongside fixes elsewhere in the standard library and toolchain.

The underlying report is unusually direct. In issue 23392, opened on January 9, Brad Fitzpatrick records a report from adamdecaf that url.Parse accepts userinfo containing newlines and non-ASCII characters. The issue describes a possible security problem when callers use the extracted credentials without further validation. It does not establish that every application using net/url is exploitable. The relevant question is what the application does with those values after parsing.

I like Go's preference for code whose operational decisions remain visible. This correction fits that preference because it gives the parser a more explicit responsibility. Understanding the exact responsibility matters more than treating every successful parse as a general security guarantee.

The authority contains its own grammar

A URL's authority can include user information before the host. The familiar form https://alice:secret@example.com/ puts alice:secret before the @ separator. RFC 3986 section 3.2.1 defines the permitted userinfo characters and percent encoding. A literal newline does not belong to that grammar.

That restriction concerns the representation of the URL. It does not define the authentication rules of every service identified by one. A syntactically accepted username can still be unknown to a server, and an accepted password can still be wrong. Separating these questions avoids giving a general parsing function authority that it cannot exercise.

The distinction also explains why rejecting raw characters and decoding escaped characters are separate operations. A parser first has to decide whether the supplied URL has a valid representation. It can then decode the data represented by escape sequences. Application code receives the decoded result and must apply whatever restrictions its next operation requires.

The standard discourages the user:password form and discusses the risks of presenting sensitive information in userinfo. That advice remains relevant even when a parser enforces the grammar correctly. Correct syntax does not make a credential-bearing URL an appropriate thing to display or retain in an application log.

The check happens before decoding

The Go 1.9.3 implementation makes the ordering concrete. parseAuthority finds the last @ and parses the host that follows it. If userinfo exists, the function checks it with validUserinfo before separating the username and password and calling unescape. Failure returns an error containing net/url: invalid userinfo.

This placement matters. Checking the original userinfo allows the library to enforce the URL representation's grammar before it converts that representation into credential strings. Moving the same check after decoding would ask a different question and could reject data that arrived through permitted percent encoding. The patch therefore cannot be summarized as a ban on every unusual character in every extracted credential.

The release's regression test supplies a compact example. It parses a URL containing literal newlines in both credential fields and expects an invalid-userinfo error. An equivalent small program makes the externally visible behavior clear.

package main

import (
    "fmt"
    "net/url"
)

func main() {
    _, err := url.Parse("http://us\ner:pass\nword@foo.com/")
    fmt.Println(err != nil)
}

For Go 1.9.3 this prints true. The input is deliberately a Go string with newline escapes, so the parser receives newline bytes. It is not a URL containing the literal text %0A. That difference is essential to the example and to the scope of the correction.

A parser error changes the caller's control flow

The useful application consequence is an earlier failure. A caller that already checks the returned error now rejects this malformed representation before using its credential fields. Code that discards the error needs its own correction. A stronger parser cannot compensate for application control flow that continues after parsing has failed.

Consider a configuration interface that accepts an endpoint URL. Successful parsing should allow the interface to proceed to endpoint-specific checks. A parsing error should prevent it from constructing a client from partially understood input. This is an inference from the parser's contract, rather than a claim about an incident or a particular client library.

The receiving operation also determines which decoded values are acceptable. If an application forwards a value into a format with restrictions on line breaks, that format's restrictions still apply. Percent decoding does not erase the receiving format's grammar. Validation therefore belongs at the point where the program knows what it is about to construct.

I prefer this division of responsibility because it leaves the application policy in ordinary code. The library rejects an invalid URL representation, while the caller decides whether the resulting endpoint and credentials are suitable for its task. Neither decision requires pretending that a URL is an authorization policy.

The release is broader than one parser fix

Andrew Bonventre's January 22 announcement lists corrections to the compiler and runtime as well as database/sql, math/big, net/http, and net/url. The security angle should not obscure that this is a maintenance release for the language implementation and its bundled libraries. Conversely, the broad package list does not explain the security mechanism, which is why the source and test are useful here.

For a program built against the corrected standard library, the relevant behavioral difference is observable at the call to url.Parse. That is a precise property to preserve in an application's own tests when it accepts externally supplied endpoint strings. The malformed userinfo case should follow the rejected-input path, and a normal credential-bearing URL should continue through whatever endpoint policy the application defines. Those two cases express different requirements and keep the parser correction distinct from the application's authentication decisions.