Kubelet TLS bootstrap separates initial access from node identity

Giving every kubelet the same credential makes initial provisioning easier while removing the identity needed to distinguish nodes. The Kubernetes 1.12 announcement on September 27 describes that operational problem as kubelet TLS bootstrapping reaches general availability. A kubelet can generate its key and request a signed client certificate through the cluster API. Operators can automate enrollment without distributing the same long-term client identity to every node.

I regard distinct node identity as a necessary part of operating a controller-based system. A cluster must distinguish which node is making a request before it can apply node-specific restrictions. Automating the issuance process helps make that distinction practical. The security question then becomes how initial access permits a certificate request and which policy allows that request to acquire a node identity.

The release also stabilizes Azure Virtual Machine Scale Sets support. Both changes concern operating node fleets, but they solve different problems. A mechanism for creating or scaling machines does not itself establish each machine's Kubernetes identity. TLS bootstrap addresses the credential provisioning step, with client certificates as its result.

Bootstrap starts with an existing trust configuration

The 1.12.0 kubelet bootstrap implementation accepts separate paths for the resulting kubeconfig and the bootstrap kubeconfig. It first checks whether the resulting configuration already exists and has usable client certificate material. When that check succeeds, it skips bootstrapping. This avoids treating every process start as a fresh enrollment.

When enrollment is necessary, the code loads the bootstrap configuration and creates a certificate-signing-request client from it. The request therefore does not begin without credentials or connection settings. Bootstrap needs an initial way to reach and authenticate to the API server. Its purpose is to exchange that initial access for the client certificate used by the kubelet afterward.

The implementation later copies the server address and certificate authority configuration from the bootstrap client configuration into the resulting kubeconfig. That preserves the chosen cluster endpoint and trust settings while changing the authentication material. It does not discover a new trust root by accepting whichever certificate happens to arrive over an untrusted connection.

The code carries the bootstrap configuration's TLS verification setting as well. Consequently, bootstrapping cannot be described as independently repairing an insecure trust configuration. Its output inherits the connection trust choices supplied to it. The enrollment process automates certificate acquisition within that configured relationship to the cluster.

The private key is local and survives a failed attempt

The bootstrap path creates a certificate file store for kubelet client credentials. It tries to reuse a private key from the current stored certificate when available and valid. Otherwise, it loads or generates a key in a temporary key file. A source comment explains why this is a separate file from the store's current path, which is a symlink managed by the certificate store.

Keeping that temporary key across unsuccessful requests is deliberate. The implementation always calls the load-or-generate helper in that path so the next startup can reuse the key if requesting the certificate fails. A restarted kubelet does not need to discard the private key associated with an enrollment attempt merely because the cluster was unavailable or the request did not complete.

After requesting the node certificate, the kubelet updates its certificate store with the returned certificate and local key. Only then does it remove the temporary key file. The final kubeconfig references the store's current path for both client certificate and key. The sequence links the issued certificate to locally held key material before switching the kubelet's ordinary client configuration to the result.

This ordering matters for operational diagnosis. A successful API request alone is not the entire bootstrap result. The process also has to store the credential and write a kubeconfig which refers to it. Filesystem errors at those later steps can prevent completion even when the cluster-side certificate work succeeded. The implementation returns those errors to its caller.

Certificate requests have an approval boundary

The 1.12.0 certificate approver makes the policy boundary explicit. It ignores requests which already contain a certificate and requests already approved or denied. For other requests it parses the certificate-signing request and checks it against recognizers. Recognizing the request's shape is separate from authorizing its approval.

The node client recognizer expects the organization system:nodes. It also requires the subject common name to begin with system:node:. DNS names, email addresses and IP addresses in the request cause that recognizer to reject it. The expected usages include key encipherment, digital signature and client authentication. These checks identify the kind of client credential the controller knows how to consider.

This is relevant to the release's distinction between client and serving certificates. A client certificate authenticates the kubelet to the API server. It does not automatically establish the certificate presented by a kubelet accepting incoming TLS connections. The announcement separately moves kubelet server certificate bootstrap and rotation to beta. Those two maturity labels should not be combined into a claim that all kubelet certificate handling is now generally available.

The self-node-client recognizer adds a further condition to the node-client checks. The requester's username must equal the requested certificate's common name. That checks continuity between the authenticated requester and the node identity being requested. It gives an existing node's request a recognizable category distinct from the more general node-client request.

The usage check also compares the number of requested usages with the expected list before checking membership. Adding unrelated usages does not leave a request eligible merely because client authentication is among them. The recognizer requires the expected set. That is a narrower condition than recognizing any certificate request whose name resembles a node, and it explains why a well-formed request can still fall outside automatic approval.

Approval uses the requester's authorization context

The approver constructs a SubjectAccessReview from the request's recorded username, UID and groups, including extra authentication information. It asks whether that subject is allowed the relevant operation on the certificate request subresource. The self-node-client and node-client recognizers use different subresource names. Policy can therefore distinguish those recognized request categories.

Only a positive authorization result leads the controller to append an approval condition and update the request's approval. A recognized shape is insufficient by itself. This prevents the existence of a syntactically appropriate node request from becoming an unconditional instruction to approve it. The authorization decision remains part of the cluster's configured enrollment policy.

If the controller recognizes a request but authorization does not allow it, the code returns an ignorable error describing the unapproved access review. If it recognizes no category, it returns without approving the request. Neither path is equivalent to issuing a signed certificate. The distinction gives other policy mechanisms room to handle requests outside the automatic approver's recognized and authorized cases.

I consider this split the useful operational property of bootstrap. Local code prepares a key and submits a request, while cluster-side policy decides whether that request qualifies for approval. Automation does not remove the identity decision. It gives that decision a specific API object and authorization check, which can be examined independently of the machine provisioning workflow.

Existing credentials and connectivity affect startup

The bootstrap configuration check loads the existing kubeconfig, constructs its transport configuration and parses its client certificates. It treats missing or expired certificate material as a reason to bootstrap again. Filesystem inspection errors can instead stop the process. The implementation distinguishes absent enrollment output from an error accessing that output, which matters when diagnosing a node that repeatedly fails to start.

Before requesting a certificate, the bootstrap path also waits for API server health using a bounded connection attempt. Failure to connect produces a warning, after which the request operation still determines its own result. A network problem at this stage is different from a request awaiting approval. Both can delay enrollment, but only the latter has reached the cluster's certificate-request workflow.

The generally available mechanism therefore covers a concrete transition. A kubelet uses its initial configuration to request a certificate for locally held key material, stores the issued credential and writes its ordinary client configuration. The automatic approver separately recognizes and authorizes node-client requests. Node provisioning can automate these operations while preserving an explicit distinction between initial cluster access and the node identity granted through approval.