Kubernetes ImagePullBackOff and ErrImagePull: find out why the image won’t pull
Fix ImagePullBackOff: read the pull error in kubectl describe, then fix the image name or tag, registry credentials, Docker Hub rate limits, network or architecture.
by OhSudo ··
Not sure this is your cause?Find it step by step: Pod not running →
Part of a learning path:Kubernetes: debugging pods
Your pod never starts:
$ kubectl get podsNAME READY STATUS RESTARTS AGEapi-6f9c7d8b5c-q8m2x 0/1 ImagePullBackOff 0 3mThe node couldn’t download the container image. ErrImagePull is the failed attempt; ImagePullBackOff means Kubernetes is waiting before the next try — longer each time, up to five minutes. Nothing inside your app has run yet, so there are no logs: the reason is in the pod’s events.
- Podscheduled on a nodestart the container
- kubelet on the nodepulls team/api:1.4.3(works)image pull(fails)
- Registryname, credentials, rate limit, network or CPU type(breaks here)
↻ ErrImagePull, then ImagePullBackOff: Kubernetes waits, longer each time (up to five minutes), and tries again.
TL;DR
kubectl describe pod api-6f9c7d8b5c-q8m2x # the Events at the bottom have the real pull error| The error contains | Cause | Section |
|---|---|---|
not found |
Wrong image name or tag | 1 |
401 Unauthorized, pull access denied, authorization failed |
Private registry, missing or wrong credentials | 2 |
429 Too Many Requests, toomanyrequests |
Docker Hub rate limit | 3 |
no such host, i/o timeout, x509 |
Network, DNS or TLS between the node and the registry | 4 |
no match for platform in manifest |
Image built for another CPU architecture | 5 |
Read the events
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Pulling 2m (x4 over 3m) kubelet Pulling image "registry.example.com/team/api:1.4.3" Warning Failed 2m (x4 over 3m) kubelet Failed to pull image "registry.example.com/team/api:1.4.3": rpc error: code = NotFound desc = failed to pull and unpack image "registry.example.com/team/api:1.4.3": failed to resolve reference "registry.example.com/team/api:1.4.3": registry.example.com/team/api:1.4.3: not found Warning Failed 2m (x4 over 3m) kubelet Error: ErrImagePull Normal BackOff 1m (x6 over 3m) kubelet Back-off pulling image "registry.example.com/team/api:1.4.3"The long Failed to pull image line is the one that matters. Find its key words in the table above.
1. Wrong name or tag
not found means the registry has no image with that exact name and tag. Typical causes: a typo, a tag that was never pushed (the CI build failed), or a short name like api:1.4.3, which means docker.io/library/api:1.4.3 — Docker Hub, not your registry.
Check that the image really exists, from your own machine:
docker manifest inspect registry.example.com/team/api:1.4.3Then fix the image in the Deployment (kubectl set image deployment/api api=registry.example.com/team/api:1.4.2, or in your manifests / Helm values).
2. Private registry: credentials
401 Unauthorized or pull access denied … may require authorization means the node isn’t allowed to pull. Pods pull with the credentials in an image pull secret:
kubectl create secret docker-registry regcred \ --docker-server=registry.example.com \ --docker-username=ci-puller \ --docker-password='<token>' \ -n productionUse it in the pod spec:
spec: imagePullSecrets: - name: regcred containers: - name: api image: registry.example.com/team/api:1.4.3Or attach it to the namespace’s service account, so every pod there uses it:
kubectl patch serviceaccount default -n production -p '{"imagePullSecrets": [{"name": "regcred"}]}'Common mistakes: the secret is in another namespace (secrets are per namespace), --docker-server doesn’t match the registry in the image name exactly, or the token expired. On managed clusters, the node’s cloud identity can pull from the provider’s own registry (ECR, Artifact Registry, ACR) — then check that identity’s permissions instead.
3. Docker Hub rate limits
429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit.Docker Hub limits anonymous pulls per IP address, and a whole cluster often shares one public IP. Authenticate the pulls with a Docker Hub account (an image pull secret for docker.io, as in section 2), use a registry mirror or pull-through cache, or copy the images you depend on to your own registry.
4. The node can’t reach the registry
no such host, i/o timeout or connection refused happen between the node and the registry, not your laptop. The node needs DNS that resolves the registry (see Temporary failure in name resolution) and an outgoing path to it — check firewalls, proxies and private endpoints.
x509: certificate signed by unknown authority means the registry uses a certificate from a CA the node doesn’t trust, often a company CA. Add that CA to the node’s trust store and restart containerd — see unable to get local issuer certificate.
To test from the node itself (if you can log in there):
sudo crictl pull registry.example.com/team/api:1.4.35. Wrong architecture
no match for platform in manifest: not foundThe image exists, but not for the node’s CPU — typically built on an Apple Silicon Mac (arm64) and pulled by amd64 nodes, or the other way around. Build a multi-platform image:
docker buildx build --platform linux/amd64,linux/arm64 -t registry.example.com/team/api:1.4.3 --push .After the fix
Kubernetes keeps retrying, but the back-off can be up to five minutes. To try right away, let the Deployment create fresh pods:
Note
About :latest. With a :latest tag (or no tag), Kubernetes pulls the image every time a pod starts, so a registry outage stops new pods. With a fixed tag like :1.4.3, a node that already has the image doesn’t pull again (imagePullPolicy: IfNotPresent). Fixed tags also make it clear what’s running.
Shell, we have a problem!
Similar, but different: CrashLoopBackOff — the image pulled fine and the container started, then exited; see Kubernetes CrashLoopBackOff. ErrImageNeverPull — imagePullPolicy: Never and the image isn’t on the node. InvalidImageName — the image reference isn’t valid (uppercase letters, a stray space). If the image is fine and a Secret isn’t, see CreateContainerConfigError. If a Helm upgrade got stuck waiting for these pods, see another operation is in progress. For the same problem with plain Docker, see pull access denied. For the commands around this, see the kubectl Cheat Sheet.
$ ./practice.sh
Practice this in the terminal
Open the Terminal Playground with this article’s scenario loaded. It is safe and runs in your browser.
Found this useful? Share it.
Found a mistake? Let us know at contact@ohsudo.com.