July 22, 2019·6 min read·engineeringdockerrootlesssecuritygpu

Docker 19.03 moves daemon privilege into an explicit operating choice

Experimental rootless operation changes the host authority of the Docker daemon. GPU device requests add another explicit resource contract, with different requirements and limitations.

Running a container process without host root authority and running the Docker daemon without it are different deployment choices. Docker 19.03 adds experimental support for the second. The July 22 release notes describe running dockerd as a non-root user alongside a new device-request API for NVIDIA GPUs. Both changes make an operating assumption explicit, the authority of the daemon in one case and the devices requested by a container in the other.

The rootless implementation is particularly interesting because its restrictions are part of the design available today. The documentation shipped with 19.03.0 describes a daemon running inside user, mount and network namespaces. It also lists facilities that do not work in this mode. Adopting it means choosing that whole arrangement, rather than changing a username on an otherwise equivalent daemon.

I value Docker for making application environments repeatable and portable. A host service's privileges are a separate part of the deployment contract. Rootless operation makes that separation more visible, which is useful even where its current restrictions prevent it from replacing an existing daemon.

The daemon enters its own namespaces

The documented entry point is dockerd-rootless.sh, invoked with the experimental flag. Its wrapper runs the daemon in its own user, mount and network namespaces. The documentation says the experimental option remains required because rootless mode itself is experimental in this release.

dockerd-rootless.sh --experimental

This is a daemon startup command, not a request to change the user inside one container. Its effects include a different default socket and different state directories. A client pointed at an ordinary system daemon would still be operating that other daemon, so client targeting is part of actually using the mode.

The tagged documentation places the socket at $XDG_RUNTIME_DIR/docker.sock by default. It places the data directory under ~/.local/share/docker and daemon configuration under ~/.config/docker. The client configuration directory remains ~/.docker. Keeping these names distinct helps explain why copying settings into the familiar client directory does not configure the rootless daemon.

The documented client example explicitly selects the rootless socket. With XDG_RUNTIME_DIR set for the user session, it has this form.

docker -H unix://$XDG_RUNTIME_DIR/docker.sock run -d nginx

The important element is the endpoint. The client remains the ordinary upstream Docker client, while the daemon answering the request is the one started in the user's namespace arrangement. Rootless operation therefore changes where the service runs and how its resources are represented without requiring an unrelated client protocol.

Subordinate identities are a prerequisite

The rootless documentation requires newuidmap and newgidmap on the host. It also requires at least 65,536 subordinate IDs in each of /etc/subuid and /etc/subgid for the user. Its sample entry assigns a range beginning at 231072 to a named account. Those are host configuration requirements, not files added to an application image.

This makes the privilege claim precise. The documentation says no setuid or file-capability binary is required except the identity-mapping helpers. The daemon operates without root, but setting up the mappings still relies on specifically permitted helper behavior. Describing the mode as requiring no privileged support at all would lose that qualification.

Distribution settings can also determine whether the namespace setup is possible. The tagged guide lists enabling unprivileged user-namespace creation on Debian outside Ubuntu and on Arch Linux. It gives a namespace-count adjustment for RHEL and CentOS 7. These requirements explain why an identical wrapper command can encounter a different host prerequisite on another distribution.

The distinction is useful for deciding which part of a deployment owns the setup. Subordinate ID ranges and kernel namespace policy belong to the host account and operating system. A Dockerfile describes the application environment. A build cannot supply missing host identity mappings merely by putting configuration files in the resulting image.

Resource controls and networking differ

The release's rootless restrictions include unsupported cgroups, AppArmor, checkpointing and overlay networking. The guide explicitly includes docker top under the cgroup restriction because it depends on the devices controller. These are missing capabilities of this experimental arrangement, not optional configuration values that recover the full ordinary-daemon behavior.

This is particularly consequential for resource management. A deployment that relies on the listed cgroup facilities cannot assume the same behavior after switching daemon mode. The documentation's diagnostic output reflects that limitation, with docker info showing none as the cgroup driver and rootless among the security options.

Storage has its own restriction. The guide identifies vfs as supported, while allowing overlay and overlay2 on Ubuntu and a few other distributions. That exception prevents a simple universal claim about either the available storage driver or the cost of changing modes. The host's implementation support is part of the answer.

For exposed TCP and UDP ports, the documented host port must be at least 1024. This changes how a service is made reachable on the host. An application can keep its internal listener while the published host-side endpoint must satisfy the rootless restriction. The port requirement is another reason that a daemon-mode change deserves an explicit deployment description.

The wrapper uses slirp4netns when installed or VPNKit as its default user-space network stack. The guide warns of possible performance overhead and describes an optional lxc-user-nic configuration for better performance. That option needs host-side setup and uses the corresponding helper, so it also changes the prerequisite set. The available networking choices are concrete implementation alternatives within this release.

The guide also treats ping routing as host configuration. It requires an appropriate net.ipv4.ping_group_range value set by root. That is another example of the distinction between running the daemon without root and preparing every host facility without administrative involvement. The unprivileged account does not acquire authority to change the host setting simply because it owns the daemon.

The documented exec directory also moves under the user runtime directory. Keeping transient execution state there and persistent image data under the home directory gives the mode a distinct filesystem arrangement. Tools that assume the system daemon location must instead target this instance.

My preference for a small final service image remains compatible with asking these host questions separately. Removing build tools from the shipped filesystem does not determine the daemon's namespace setup or network implementation. Rootless mode changes the runtime environment responsible for starting that filesystem.

Device requests describe a different resource boundary

The same release adds DeviceRequests to HostConfig for NVIDIA GPU support. The implementation proposal in pull request 38828, merged in March, describes a deliberately generic request model while acknowledging that the implementation initially hard-codes NVIDIA support. That proposal separates capabilities used for matching devices from options intended to configure them.

That distinction gives a device request more meaning than an arbitrary set of driver settings. A driver can advertise capabilities, and the request can identify the capabilities it requires. The proposal uses gpu as a shared capability and identifies NVIDIA capabilities such as compute and utility. It describes broader vendor and scheduling uses as future intentions, not support already supplied by this patch.

The author also states that the NVIDIA driver does not currently use the options field. An example such as limiting GPU memory appears in the proposal as something a future implementation could do. It would be wrong to turn that example into a Docker 19.03 resource-limit guarantee. The request data structure accommodates more than the initial driver implements.

Rootless operation and GPU requests therefore address different relationships. The former constrains how the daemon operates under a host identity. The latter describes a container's request for a device capability. Their appearance in the same release does not establish that every device use case works with every rootless configuration, especially given the tagged rootless guide's explicit resource-control restrictions.

The operating mode belongs in the deployment description

Docker 19.03 makes it possible to select a daemon arrangement with reduced host authority, but the release calls that arrangement experimental and documents its limits. A meaningful description names the daemon endpoint and the host mappings it needs. It also includes the storage and network choices that determine how the service actually runs.

The GPU addition follows the same need for precise scope in its own area. Its request model supports capability matching, while the initial implementation targets NVIDIA and leaves some generic fields unused. Both changes are useful because their contracts can be inspected directly. The tagged rootless documentation and the device-request proposal give concrete boundaries for what this release supplies.