March 19, 2019·6 min read·engineeringistioscalabilityconfigurationsidecars

Istio 1.1 makes configuration scope a scaling tool

Istio 1.1 addresses the cost of operating larger meshes through scoped proxy configuration and changed defaults. The new Sidecar resource connects namespace boundaries to the amount of work performed by Pilot and Envoy.

A proxy does not need every service in a mesh merely because all those services exist. Computing and distributing unnecessary configuration consumes control-plane resources and increases the configuration each proxy has to handle. The Istio 1.1 release on March 19 addresses this cost through a new Sidecar resource that can restrict a workload's configured destinations. Its scaling story is partly a story about defining which information belongs at each proxy.

The announcement is candid about the motivation. Larger production installations encountered sidecars consuming too many resources and adding too much latency, while Pilot demanded excessive control-plane resources. The release concentrates on efficiency alongside namespace isolation and multicluster operation. These are related problems when the amount of configuration distributed across a mesh grows with the services it contains.

I see that resource cost as part of evaluating a service mesh, because traffic policy adds another distributed system to operate. Istio's uniform configuration can be useful across a fleet, but the benefit depends on what the control plane must compute and what every workload must carry. Scope is a concrete way to address that overhead.

Sidecar configuration limits the information a proxy needs

The 1.1 change notes explain that the Sidecar resource controls proxies attached to workloads within a namespace. In particular, it can limit the services to which a sidecar sends traffic. The notes connect that limit to less configuration being computed and transmitted, improving startup time, resource consumption and control-plane scalability. They recommend a Sidecar resource per namespace for large installations.

The mechanism is more useful to understand than an unspecific claim of improved performance. Restricting the destinations reduces the configuration relevant to a particular proxy. Pilot has less information to prepare for that proxy, and the configuration transfer is correspondingly smaller. This addresses the shape of the workload imposed on both sides of the configuration connection.

The same resource also exposes controls for ports, protocols and traffic capture. Those controls make it a description of proxy behavior, not merely a performance hint. A scope that omits a destination the workload requires would express the wrong configuration. The scaling benefit follows from an accurate declaration of the services the workload actually needs.

This approach places some knowledge with the person configuring the namespace. A mesh-wide default can be convenient when dependencies are not explicitly described. A scoped configuration requires those dependencies to be represented. The exchange is worthwhile when the reduction in distributed configuration matters, but it is still a change in the information the operator must maintain.

Visibility and reference scope are separate concepts

The release also introduces exportTo for controlling which namespaces can reference services. The change notes identify support on ServiceEntry and VirtualService resources, as well as through the networking.istio.io/exportTo annotation on Kubernetes Services. A service owner can therefore define where configuration is visible instead of assuming every namespace should refer to it.

Gateway references receive an explicit namespace qualification as well. The notes describe ambiguity when several namespaces define a virtual service for the same hostname. A namespace-qualified host reference resolves which configuration is intended. The corresponding capability is available for Sidecar egress configuration. The configuration model can distinguish identical host strings owned in different namespaces.

These changes should be understood according to their stated responsibility. Reference visibility and proxy configuration scope describe which configuration participates in a workload's traffic model. The release separately lists authorization capabilities. A configuration boundary should not be interpreted as an automatic substitute for the security policy that decides which traffic is allowed.

I favor explicit namespace boundaries because they make responsibility easier to understand. Here they also prevent configuration from being treated as globally relevant by default. That combination is useful in a shared platform where several owners contribute traffic rules through the same control plane.

Changed defaults remove work from the request path

Istio 1.1 disables Envoy sidecar access logs by default to improve performance. It also turns policy checks off by default for most scenarios, with instructions for enabling them where required. The notes distinguish these changes from the presence of security capabilities elsewhere in the release. An operator must know which optional policy mechanism the installation actually invokes.

These defaults have observable consequences. An installation using the default log behavior will not produce the same access-log stream as one with logging enabled. A policy setup that depends on the disabled checks needs the relevant enforcement option enabled. Lower overhead obtained by changing default work is different from retaining identical work and making its implementation faster.

Telemetry receives other reductions. The release cuts default Envoy-generated statistics, adds load shedding to Mixer and improves the Envoy-to-Mixer protocol. Those changes address collection volume and the system processing it. The absence of a previously collected statistic can therefore follow a deliberate default change, rather than indicating that the proxy has stopped handling traffic.

The announcement points readers to performance testing but does not establish a universal resource saving for every mesh. The useful conclusions here are mechanistic. Fewer configured destinations reduce configuration, disabled logging removes default log production, and fewer default statistics change collection work. Their effect depends on the installation's actual traffic and enabled behavior.

Galley separates configuration ingestion from its consumers

The release notes describe Galley as the primary configuration ingestion and distribution mechanism. It validates, transforms and distributes configuration, insulating other Istio components from Kubernetes details. Components interact with it through the Mesh Configuration Protocol. That defines another boundary within the control plane, separating configuration acquisition from the components using the resulting state.

The release also adds istioctl validate for offline validation of Kubernetes resources containing Istio configuration. A separate verify-install command checks an installation against a supplied installation YAML file. These tools ask different questions. One examines resource configuration before it is submitted, while the other examines the deployed installation against an expected description.

Separating those questions is helpful for an operator. A valid configuration does not prove that every expected component is installed, and the presence of components does not prove that a particular resource expresses the desired traffic behavior. The new commands provide distinct entry points into these parts of operating the system.

Installation packaging changes too. CRDs move into an istio-init chart, which the release notes say preserves custom-resource data continuity during upgrades and allows installation to evolve beyond Helm. This gives the definitions that establish the mesh's resource types a lifecycle distinct from the rest of the packaged components. Existing configuration needs continuity while the software consuming it changes.

Security and multicluster improvements retain their own configuration

Istio 1.1 adds identity provisioning through SDS, with on-node key generation and dynamic certificate rotation without restarting Envoy. The release also adds authorization for TCP services and for list-valued JWT claims such as groups. These capabilities extend the security mechanisms available to an installation, while still requiring the administrator to configure the intended enforcement behavior.

The change notes add support for Kubernetes HTTP readiness and liveness probes when mutual TLS is enabled. They also introduce organization- or cluster-specific trust domains. Both concern integrating workload identity into the environment where the mesh runs. The former addresses platform health checks, while the latter changes how identities name their trust domain.

Authorization configuration also changes scope. The release replaces RbacConfig with ClusterRbacConfig to give the resource the correct cluster scope and provides a migration path for existing configuration. That is another example of the distinction between a feature remaining available and its declaration staying unchanged. An installation moving to the new release needs the replacement resource where the previous type expressed its authorization setup.

Multicluster operation gains topologies using ingress gateways without requiring pod-level VPNs. The notes describe per-cluster control planes for high availability and namespaces spanning clusters. Locality-aware routing supports preferring services in the same locality before choosing other locations. These options extend the deployment model without making every installation require the same network topology.

The release explicitly includes incompatible behavior changes and recommends a manual upgrade of control and data planes. That qualification is consistent with the scope of the changes above. Namespace visibility, default logging and policy checks all affect what an existing installation does. Istio 1.1's scalability improvements are most understandable when those changes are treated as configuration and workload decisions, with the Sidecar resource reducing the information each proxy needs to receive.