November 2, 2020·6 min read·engineeringdockerregistryoperations

Docker Hub pull limits make registry identity operational

The new limits count manifest requests against authenticated users or anonymous IP addresses. That changes how shared infrastructure and automated image retrieval depend on registry authentication.

A public container image can be available to everyone while the service distributing it still limits how often a particular client retrieves it. Docker Hub begins gradually enforcing that distinction today. Docker's 2 November announcement sets eventual limits of 100 anonymous pulls and 200 authenticated free pulls per six hours. The rollout is gradual, with full application planned over the coming weeks.

The change makes registry identity consequential for routine automation. Public access to an image and an unlimited allowance for retrieving it are separate properties. A build system may have had no reason to authenticate when all its images were public. Under the new policy, the identity associated with those requests determines the applicable allowance.

I value Docker because a built image can move between environments without reconstructing its contents on each machine. That benefit depends on distribution infrastructure. Pull limits make that dependency explicit, particularly for machines that repeatedly start from an empty local image store. The image format remains useful, while retrieving the artifact acquires a more visible operational constraint.

Docker counts manifest requests

The documentation available at the start of enforcement defines a pull as up to two GET requests to registry manifest URLs. A normal image pull makes one manifest request, while a multi-architecture pull makes two. HEAD requests do not count. This definition is more specific than a general limit on every HTTP request or every downloaded layer.

That distinction matters when estimating consumption. Counting containers started does not directly count manifest retrievals. Similarly, counting downloaded bytes does not give the policy's unit. An engineer needs to identify the requests that obtain manifests, including requests made by the tools involved in a build or deployment. The documented accounting unit is the starting point for understanding the allowance.

The multi-architecture rule also avoids a misleading reading of the two-request definition. The document describes two manifest requests as one pull for that case. It does not simply charge every HTTP request as a separate image pull. The initial retrieval can identify the appropriate platform manifest before the client obtains the manifest describing that platform's image.

This is a registry-level policy. It does not alter the instructions inside a Dockerfile or the filesystem contents of the image. Its effect occurs when a client needs Hub to supply image information. That can interrupt the workflow surrounding an otherwise unchanged application, because successful image execution depends on obtaining the artifact first.

The gradual rollout means the final published allowance and the limit observed during introduction may differ. A successful series of anonymous requests today therefore does not disprove the announced policy. The final values describe the intended enforced state. Response information describes what the service is applying to a particular request during the transition.

Anonymous traffic shares an IP-based identity

Docker documents anonymous enforcement by IP address. Authenticated requests use the pulling user's entitlement, including the highest entitlement available through the user's account or organization membership. Logging in is necessary to use that authenticated entitlement. The documentation states that ordinary limits depend on the person pulling, rather than the owner of the image.

A shared outbound address therefore has a practical consequence. Several independent anonymous clients behind that address can contribute to the same IP-based allowance. A developer looking only at one machine's activity may not see all the requests associated with the public address. The accounting identity is broader than that machine when network infrastructure combines its traffic with other clients.

Authentication changes which identity the service can use. It does not mean a free account has unlimited pulls, since the announced authenticated free allowance is still finite. The relevant change is both the larger stated allowance and its association with the authenticated user. Sharing a single identity across many jobs can still concentrate those jobs' demand under that identity.

Those consequences follow from the documented accounting model without requiring assumptions about a particular company's network. A machine's local user, the account running a CI job and the account presented to Hub need not be the same identity. The registry can apply only the identity supplied through its own request path.

Authentication belongs to the component doing the pull

The historical guide gives different authentication routes for different execution environments. Desktop supports signing in through its menu. Standalone Engine users can use docker login. Swarm needs registry authentication passed to services, and Kubernetes needs image-pull credentials. The guide also points CI users toward their platform's registry-login integration.

These are distinct places to configure access because different components retrieve images. Logging into a registry in an interactive development session does not by itself establish which credentials a separate deployment system sends. A successful local pull confirms the local path. It does not establish the identity of every request made when an automated system creates a new environment.

For a deployment failure, that distinction narrows the investigation. The question is which component requested the manifest and how it authenticated. Looking only at the application's runtime logs can miss a failure that happened before its image was available. The application's dependencies may be unchanged while the image-retrieval path begins returning a policy error.

I prefer to treat registry authentication as part of the environment definition for this reason. It determines how an external service accounts for requests essential to starting the application. Leaving it implicit makes a workflow depend on anonymous network identity, which can include activity outside the immediate build or deployment.

A pull-limit failure has a specific response

The documentation says an over-limit manifest request receives HTTP 429 with a message identifying the pull-rate limit and linking to information about increasing it. The same condition can appear in the Docker CLI or Engine logs. This gives the failure a recognizable source, distinct from an application error after startup.

Hub also has an overall request limit to protect its infrastructure. The historical page describes that separate limit as applying across requests and account levels, with a simple 429 response. The longer pull-limit message distinguishes the image-pull policy from that overall protection. A status code alone is therefore insufficient to identify which limit the client reached.

Valid manifest responses can include RateLimit-Limit and RateLimit-Remaining headers. Docker describes those headers as best effort, with small variations possible. The documented example is equivalent to the following values, where the window is expressed in seconds.

RateLimit-Limit: 100;w=21600
RateLimit-Remaining: 76;w=21600

The example means a limit of 100 within a six-hour window, with 76 pulls remaining. It is an illustration of the protocol information, not a measurement from an environment I operate. Reading the remaining allowance gives a more direct account of the policy state than estimating it from container starts or layer-download progress.

The guide's example diagnostic manifest request uses GET and consequently consumes allowance. Its accounting section separately says HEAD does not count. That difference matters for monitoring. Observing a limit should not be assumed to be free of accounting effects merely because the request was made for diagnostics rather than to start a container.

The open-source exemption belongs to a namespace

Today's announcement also creates an exception for approved noncommercial open-source projects. Docker suspends the pull restrictions for users retrieving images from approved namespaces. Qualification requires the repositories in the publisher's namespace to meet the program's criteria, including public, noncommercial status and distribution under an OSI-approved license. Publication of source code alone is not the approval process.

Docker describes an application and review process, with an approved waiver lasting a year and eligibility reviewed for renewal. A publisher can have other namespaces that do not qualify. That is a different mechanism from the ordinary user's authenticated allowance. The approved publishing namespace creates the exemption for its images, rather than requiring every downstream user to obtain an individual exception.

The distinction is useful when assessing a dependency on a widely shared image. Its license and its program status are related through the qualification criteria, but they are not interchangeable. An image being open source does not automatically establish that its namespace has received the waiver. The ordinary pull policy and the approved-namespace exception need separate evidence.

The operational change is consequently precise. Hub can account for anonymous requests by address and authenticated requests by user, while approved open-source namespaces have a documented exemption. A working image-distribution path now needs to account for those identities as well as the image name. That relationship determines which allowance applies when automation requests the next manifest.