July 18, 2019·6 min read·engineeringkubernetesapiupgradesworkloads

The Kubernetes 1.16 API removals require more than new version strings

July's deprecation notice gives operators time to migrate before Kubernetes 1.16. Selectors and rollout defaults make the workload API changes a behavioral migration as well as a change of endpoint.

A deployment file can keep working for years while the API version written at its top becomes obsolete. The failure arrives when the server stops serving that endpoint. Vallery Lancey's July 18 notice about API removals in Kubernetes 1.16 gives operators advance warning of that transition. The release is expected in September, and several older workload API versions will no longer be served. Updating the version string is part of the migration, but changed selectors and defaults can also change the meaning of a declaration.

The notice distinguishes removal of an API endpoint from loss of persisted objects. Existing data remains retrievable and updatable through the replacement versions. This is why the work belongs in manifests and clients as well as upgrade planning. A resource may still exist while the particular request used by a deployment tool stops working.

I value a declarative API when its behavior remains understandable across upgrades. Here that requires reading the target version's contract rather than assuming that an accepted old manifest expresses everything its author intended. Defaults supplied by the old endpoint are part of that history, even when they never appeared in the file.

The replacement APIs already exist

DaemonSets, Deployments, StatefulSets and ReplicaSets move to apps/v1, which the notice says has been available since Kubernetes 1.9. The deprecated versions differ by resource. Deployment and ReplicaSet removals include extensions/v1beta1, apps/v1beta1 and apps/v1beta2. DaemonSet removal concerns extensions/v1beta1 and apps/v1beta2, while StatefulSet removal concerns the two older apps versions.

NetworkPolicy moves from extensions/v1beta1 to networking.k8s.io/v1, available since 1.8. PodSecurityPolicy moves from extensions/v1beta1 to policy/v1beta1, available since 1.10. The target API for that security resource remains beta. The migration therefore follows the replacement contract identified for each resource, without implying that all replacements have the same maturity label.

Because those endpoints already exist, this work can precede installation of 1.16. A client can learn to use the replacement while its current server still accepts the old version. The July warning creates an opportunity to separate client and configuration changes from the server upgrade that makes them mandatory. That reduces the amount of behavior changing at one operational boundary.

Required selectors make ownership explicit

For the workload resources named in the notice, apps/v1 requires spec.selector and makes it immutable after creation. The migration guidance says to use the existing template labels as the selector for a seamless upgrade. This is a substantive field requirement, not a spelling change in apiVersion.

A selector describes which pods belong to the workload's selection. Making it explicit forces the resource declaration to preserve that relationship. Immutability also means that later edits cannot treat the selector as an ordinary mutable deployment parameter. A migration that chooses a new value merely because it looks cleaner would change the intended relationship and encounter the target API's restriction.

This fragment illustrates a matching selector and pod-template label inside an otherwise complete Deployment. It does not provide a full workload, because the template would also need its container specification.

apiVersion: apps/v1
kind: Deployment
spec:
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example

The particular label is illustrative. For an existing workload, the important input is the labels it already uses, as the notice recommends. Introducing an explicit selector should preserve the intended selection instead of silently choosing a different set of pods. This is one reason a mechanical version-string substitution cannot establish that a migration is complete.

Omitted rollout settings have versioned behavior

The Deployment changes include a default progress deadline of 600 seconds, where extensions/v1beta1 had no deadline. Revision history defaults to ten retained revisions in the replacement, compared with two in apps/v1beta1 and retaining all in extensions/v1beta1. These differences affect the behavior attached to fields that a short manifest may omit.

Rolling-update defaults also change. The notice says maxSurge and maxUnavailable default to 25 percent in the replacement, compared with one in extensions/v1beta1. The apps/v1 definitions in Kubernetes 1.15 explain how these values work. Unavailability is the maximum number of pods that can be unavailable during an update. Surge is the maximum that can be scheduled above the desired replica count.

Those definitions specify opposite rounding directions for percentages. Unavailability rounds down, while surge rounds up. For an illustrative desired count of ten replicas, 25 percent therefore corresponds to two unavailable pods and three extra pods. That arithmetic follows the API's declared rules. It shows why the new defaults cannot be evaluated without considering the workload's scale.

Explicit values can preserve a chosen operational policy. A workload that intentionally permits one extra pod and one unavailable pod can state those absolute limits. A workload intended to scale its update allowances with replica count can use percentages. The migration requires deciding which behavior was intended, because the previous API version may have been making that decision implicitly.

DaemonSets and StatefulSets have a related change in update strategy. Their replacement defaults to RollingUpdate, while the older versions identified in the notice defaulted to OnDelete. A file that omits the strategy can therefore describe a different update process after conversion. The notice also identifies removal of DaemonSet templateGeneration and Deployment rollbackTo, so obsolete fields need their own treatment.

The target API also distinguishes readiness from availability in Deployment status. Its definitions describe ready replicas as pods that are ready, while available replicas have remained ready for at least minReadySeconds. Updated replicas match the desired template. These distinctions help explain why examining a rollout involves more than comparing a desired count with a single observed number.

The progress condition has its own definition. Creating or adopting a new ReplicaSet and scaling new pods up or old pods down count as progress. The API comments say progress is not estimated for paused Deployments or where no progress deadline is specified. A newly defaulted deadline therefore enters an existing reporting mechanism with defined conditions.

Clients can outlive the manifests they apply

The notice explicitly includes custom controllers and integrations in the migration. Third-party systems such as continuous-delivery software and ingress controllers also need to call supported APIs. Editing a repository's YAML cannot update a separately installed executable whose client still requests a removed endpoint. The relevant inventory includes anything that creates, reads or watches the affected resources through those versions.

This distinction matters for determining whether a quiet period proves readiness. A tool that runs only during deployment may make no request until its next invocation. A configuration conversion can succeed while that tool still selects an old path. Removing the server endpoint exposes the dependency when the client next uses it, so readiness has to include the integrations that act intermittently.

Kubectl provides a conversion command, and the notice gives an example converting an older Deployment to apps/v1. It also warns that conversion may choose non-ideal default values. The generated output can help produce the target representation, but it cannot recover an operator's unstated intention about history retention or rollout capacity. Those are choices that must be assessed against the fields described above.

I prefer explicit rollout settings where their effect matters to availability. That is an opinion about making the configuration legible, rather than a requirement that every field be written out. The version transition is a useful moment to identify which omitted values were acceptable only because of the old API's behavior.

The server can expose remaining dependencies before the upgrade

The notice supplies an API server --runtime-config setting that disables the affected old resources in a 1.15 server. This simulates their removal before the forthcoming release. It disables the two older apps API versions and the relevant resource paths under extensions/v1beta1, giving integrations an environment in which requests to those endpoints fail.

That mechanism addresses a concrete compatibility question. Can the manifests and installed clients perform their work when those endpoints are absent? A successful conversion of stored configuration answers only part of it. Exercising the same deployment and controller paths against the disabled endpoints reveals callers whose dependency remained hidden while the old server continued accepting requests.

Existing persisted data remains accessible through the replacement APIs, so the transition does not require treating every workload as a new object. The immediate work is preserving intended field behavior and changing the clients that address those objects. With the replacement APIs already available, those changes can be completed against the current cluster version before the release removes the old routes.