April 5, 2019·6 min read·engineeringistiosecurityenvoyhttp

Istio's April security patch aligns proxy and backend request parsing

Istio 1.0.7 fixes Envoy vulnerabilities involving URI paths and NUL characters in HTTP headers. The important boundary is whether policy and the backend interpret the same request as the same operation.

A proxy can enforce a perfectly consistent rule against the wrong interpretation of a request. If the backend resolves the request differently, the policy decision may allow an operation the operator intended to prohibit. The April 5 Istio 1.0.7 announcement describes two Envoy vulnerabilities involving this boundary. One concerns unnormalized HTTP URI paths. The other concerns incomplete validation of HTTP/1.1 header values containing NUL characters.

The release incorporates fixes for CVE-2019-9901 and CVE-2019-9900 respectively. Istio also releases 1.1.2 for installations on its newer branch. These patches matter to authorization, routing and rate limiting when those decisions depend on paths or headers. The same request can pass through several interpreters, so the representation used at each decision point becomes security-relevant.

I value moving traffic policy into shared infrastructure when its enforcement is consistent across applications. That benefit depends on the proxy and application agreeing about what the request means. These vulnerabilities show why policy configuration and the software parsing its inputs are both part of the enforcement system.

Path matching needs a common representation

The announcement says Envoy did not normalize HTTP URI paths before using them. A backend can resolve a specially crafted path differently from the proxy's path-matching logic. Where a rule uses prefix matching, that difference can expose a path that should have required another authorization decision or route. The advisory makes the backend's HTTP implementation part of the affected conditions.

The Envoy 1.9.1 release history describes the fix as normalizing paths before routing or layer-seven data-plane processing. The order is significant. Applying normalization after a policy or route decision would not make the earlier decision use the backend's intended representation. The normalized path needs to be available when those operations evaluate it.

Normalization and authorization have different jobs. Normalization supplies a consistent representation for path-based processing. Authorization still decides whether the identified request is permitted. Installing the fix does not define an application's access rules. It corrects the parsing behavior on which existing rules depend, reducing disagreement between the proxy's interpretation and the backend's resolution.

The Istio announcement identifies Mixer prefix matching, Istio authorization policies and routing rules as relevant configurations. A shared proxy can use the same path information for several functions, so a parsing discrepancy can affect more than a single authorization filter. An operator examining the exposure needs to consider which decisions use that representation and how the backend handles the corresponding request.

The fixed capability has different defaults in Envoy and Istio

Envoy 1.9.1 introduces the path-normalization fix with normalization disabled by default. Its release history names a connection-manager setting and a runtime option as ways to configure it. Merely identifying that upstream version is therefore insufficient to establish that normalization is active in an arbitrary Envoy deployment.

Istio's patched builds make a different choice. The announcement explicitly says the Envoy embedded in Istio 1.0.7 and 1.1.2 enables path normalization by default. This is an important distinction between the upstream component's capability and the integration's effective configuration. The Istio remediation combines the necessary code with the enabled behavior.

That also explains why the release advises updating Istio immediately. The relevant artifact is the embedded proxy implementation used by the mesh, with its security defaults. A version check against a separately installed Envoy executable would not establish the behavior of the proxies handling mesh traffic. The component enforcing the route or policy must be the corrected implementation.

I prefer security guidance that states the effective setting alongside the component version. Here the announcement does so explicitly, preventing the upstream default from being mistaken for Istio's default. It gives operators a concrete expectation for the patched releases.

Header validation has a separate failure mode

CVE-2019-9900 concerns NUL characters embedded in HTTP/1.1 header values. Envoy's release notes say the fix rejects such headers. The Istio announcement describes the vulnerability as another route to inconsistent interpretation, potentially allowing unauthorized resources, circumventing rate limits or reaching an otherwise unexposed upstream system.

Rejecting a malformed header at the proxy prevents it from participating in later decisions under conflicting interpretations. This differs from rewriting a policy rule around one observed payload. The parser must establish which header values are acceptable before the rest of the request-processing system uses them. Otherwise several policy mechanisms can inherit the same underlying ambiguity.

The announcement says current information limits this header issue to HTTP/1.1 traffic. That condition is separate from the path issue. A deployment whose structure excludes HTTP/1.1 may not meet the header exploit's conditions, but that observation alone does not resolve the URI normalization problem. The two CVEs share a release response while addressing different input-processing behavior.

The advisory also notes that some backend servers may reject NUL characters with a bad-request response. That is useful context but does not establish safety for every backend. The announced impact depends in part on the backend implementation, so assuming one server's rejection behavior applies to every service would exceed the evidence.

Existing logs offer uneven evidence

The header exploit is not trivially detectable by searching Envoy access logs for a NUL. The announcement explains that file-based logging and gRPC access logging use the c_str() representation for header values. That representation prevents the proposed simple scan from supplying reliable evidence about the embedded character.

Instead, the release suggests looking for inconsistencies between actual Envoy routing and the logic intended by RouteConfiguration. External authorization and rate-limit services can check headers for NUL characters, and backend logs may contain useful evidence. These are different observation points in the request path. Their usefulness depends on the data they retain and how they represent it.

Path evidence is different. The announcement says Envoy access logs contain the unnormalized path, making suspicious patterns and requests inconsistent with intended configuration available for inspection. It also identifies external authorization, rate-limiting services and backend servers as places where the unnormalized path can be inspected. The representation preserved in the log determines the kind of comparison an operator can make.

This distinction matters when interpreting an absence of suspicious records. A log format that cannot expose the relevant header character does not prove that no such input arrived. A path log may provide different evidence because it preserves the unnormalized value. Detection needs to follow the actual representation rather than treating all access logging as interchangeable.

The routing evidence and policy evidence answer different questions. A route log can show which upstream Envoy selected, while a policy rule describes which request attributes should select or permit it. Comparing the two is useful when the original path representation could conceal the backend operation ultimately reached. The announcement's suggested inconsistency checks depend on retaining that relationship between intended configuration and observed processing.

The same reasoning applies to rate limiting. If a crafted request reaches a resource through a representation the limiting logic does not recognize as that resource, the configured limit may not protect the intended operation. The advisory names that possibility explicitly. Correcting request interpretation supports the existing limit by making the decision operate on the representation used for subsequent routing, without changing the operator's chosen rate policy.

The patch preserves the existing release line

The announcement describes Istio 1.0.7 as built from the same source as 1.0.6 with the minimum Envoy patches needed to address these vulnerabilities. It gives the equivalent relationship between 1.1.2 and 1.1.1. That makes the security update available without requiring a 1.0 installation to adopt the broader behavior changes of the 1.1 release.

Affected 1.0 releases run through 1.0.6, while affected 1.1 releases include 1.1 and 1.1.1. Earlier unsupported release series will not receive patches according to the announcement. Operators on those versions need a supported release containing the fixes. The supported-branch patches provide the corrected proxy behavior while retaining the surrounding release's implementation.

The operational objective is therefore specific. Requests must encounter a proxy that rejects the malformed header values and normalizes paths before making routing or policy decisions. Istio 1.0.7 and 1.1.2 supply that behavior with normalization enabled. Existing policy rules still express the desired access decision, and logs need interpretation according to what they preserve about the original request.