April 4, 2018·4 min read·engineeringkubernetessecuritystorage

The subPath fix holds the file open through mount preparation

The April technical explanation of CVE-2017-1002101 describes a filesystem race. The Linux implementation uses file descriptors and an intermediate bind mount to preserve the validated target.

A relative path inside a volume can resolve outside that volume when one of its components is a symlink. Validating the path once is insufficient if a container can change it before a privileged process uses it. The April 4 explanation of the Kubernetes subPath vulnerability describes this problem in CVE-2017-1002101, disclosed on March 12. The repair has to preserve the target between validation and the container runtime's mount operation.

I regard this as a significant qualification to declarative infrastructure. A pod specification can express a legitimate relative path while the filesystem beneath that path changes independently. Kubernetes has to enforce the storage boundary at the point where it performs privileged work. Checking the text of the declaration cannot establish which filesystem object a later mount will use.

Mount preparation crosses a trust boundary

The original fix in pull request 61044 changes the kubelet's mount preparation along with the mount utility implementations. A pod's subpath still has to be relative and pass backstep validation. Kubelet also resolves the volume's base path before joining the subpath. These checks handle different questions. A relative path can avoid parent-directory syntax and still encounter a symlink controlled by a writer to the volume.

The fix introduces PrepareSafeSubpath as an operation with a returned host path and an optional cleanup action. That return shape recognizes that safe preparation can allocate resources whose lifetime extends beyond the function call. Preparing a mount source is no longer just producing a string. The caller needs to use the prepared source and respect the associated lifetime.

The Linux mount implementation in Kubernetes 1.10.0 makes this explicit. doBindSubPath first evaluates symlinks, cleans the resulting path and verifies that it lies within the volume path. Its comment calls this an early check before the final check. The code does not treat successful symlink evaluation as sufficient authorization to hand the original path directly to the runtime.

That ordering is essential to understanding the repair. A user could replace a directory after the early check. Another lookup by pathname would then traverse the replacement. The implementation needs a way to refer to the checked object which does not depend on a later traversal of the same mutable path components. The intermediate mount is built from an open descriptor for that reason.

Traversal proceeds through directory descriptors

The helper doSafeOpen calculates the path relative to the volume's base and splits it into segments. It opens the base directory with no-follow flags, then opens each segment relative to the previous directory descriptor using Openat. At each step it also checks that the accumulated path remains within the base. Failure to open a segment aborts preparation.

Each successful child descriptor becomes the parent for the next step. The previous parent descriptor closes after the child opens. This replaces repeated traversal from a mutable pathname with a sequence anchored in already opened directories. The implementation's stated assumption is that the base is controlled by Kubernetes and cannot itself be a symlink.

At the end, the helper returns the final descriptor without closing it. Its cleanup logic deliberately transfers ownership of that descriptor to the caller. doBindSubPath keeps it open while preparing the bind mount and defers closing it until the function returns. This is the interval in which a pathname-only implementation would otherwise lose the connection between checking and using the target.

The source for the intermediate bind mount is a proc filesystem path identifying that open descriptor. Its construction uses the kubelet process identifier and descriptor number. Mounting then associates it with a target beneath the pod's subpath directory. The prepared host path returned to the caller is this intermediate target. A runtime receives a source already associated with the opened object.

The prepared mount has its own lifecycle

The target directory hierarchy includes the pod directory, volume name and container name. A volume mount index selects the final entry. This gives separate mount entries within the kubelet-managed hierarchy. The code checks whether the target is already mounted and returns it when preparation has already succeeded, which supports repeated calls without layering another mount onto the same target.

Preparation also handles the difference between a directory and another filesystem object. It creates a directory target for directory sources and an empty file target otherwise. The source comment explains that bind mounting supplies the target object's type. A safe mount implementation must handle file subpaths as well as directory subpaths because both require the same boundary enforcement.

The same source file contains SafeMakeDir, which creates missing directories through descriptor-relative operations. It reopens newly created segments with no-follow flags before continuing. This extends the concern beyond existing mount sources. Creating a missing subdirectory also traverses names which another writer may change, so preparation needs controlled traversal while constructing the path as well as while opening it.

An error triggers deferred cleanup unless the operation has marked itself successful. Cleanup checks whether the intermediate target is mounted, unmounts it when necessary and removes the target nonrecursively. Another helper removes empty directories upward toward the pod directory, stopping when it encounters remaining contents. The implementation avoids recursively deleting a mounted tree as a substitute for explicit unmounting.

On Linux, the preparation method returns no container-start cleanup action. The code says the bind mount remains until subpath cleanup. That differs from closing the file descriptor, which can happen once the intermediate mount exists. The mount and descriptor have distinct lifetimes, even though both participate in preserving the target during preparation.

I find the cleanup work an important part of the fix's engineering cost. Preserving a checked object adds persistent mount state which the kubelet must later remove. The security property depends on establishing that state before runtime startup, while normal volume teardown depends on clearing it in the correct order. The Linux cleanup loop walks the recorded subpath hierarchy and unmounts individual entries before removing their containing directories.