In this post, we will see a few ways to control and enforce the container image download behavior of Kubernetes(kubelet). We will also see the various examples with different Image Pull policies used.
The Default behavior
When a pod creation is requested, it will be scheduled on the most suitable node by the Kube-scheduler—the kubelet running on that node would query the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet pulls the image with the resolved digest and uses it to launch the container.
Changing the default behavior
You may change the default behavior by modifying the Pod manifest file’s .spec.contianers[*].imagePullPolicy field with one of the following three values in column 1 of the below table:
| Image Pull Policy | Is this Default Policy | Behavior |
| Always | YES | When a pod creation is requested, it will be scheduled on a node by default. Then, the kubelet running on that node queries the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet pulls the image with the resolved digest and uses it to launch the container. |
| IfNotPresent | NO | The image is pulled only if it is not already present locally. |
| Never | NO | The kubelet does not try fetching the image. If the image is somehow already present locally, the kubelet attempts to start the container; otherwise, the startup fails. See pre-pulled images for more details. |
The following examples demonstrate how changing the image pull policy would change the way image downloading would work in Kubernetes.
Example-1: Using ‘Never’ as an image pull policy when an image is absent on the scheduled node
| POD-MANIFEST | Initial State of the image on the Scheduled node (Eg: kube-worker-1) | Image state after on the scheduled node after pod creation request(Eg: kube-worker-1) | Pod Status after Creation request |
apiVersion: v1 name: my-pod imagePullPolicy: Never resources: {} | IMAEGE IS NOT PRESENT ON THE NODE | IMAGE IS NOT PRESENT ON THE NODE | NAME READY STATUS RESTARTS AGE |
sudo crictl images |grep -i nginx | sudo crictl images |grep -i nginx | Events: |
Example-2: Using ‘Never’ as a pull policy when an image is already present on the scheduled node
| POD-MANIFEST | Initial State of the image on the Scheduled node (Eg: kube-worker-1) | Image state on the scheduled node after pod creation request(Eg: kube-worker-1) | Pod Status after Creation request |
apiVersion: v1 name: my-pod imagePullPolicy: Never resources: {} | IMAEGE IS PRESENT ON THE NODE | IMAGE IS PRESENT ON THE NODE | kubectl get pod |
sudo crictl images |grep -i nginx | sudo crictl images |grep -i nginx | Events: |
Example-3: Using IfNotPresent as an image pull policy
| POD-MANIFEST | Initial State of the image on the Scheduled node (Eg: kube-worker-1) | Image state after on the scheduled node after pod creation request(Eg: kube-worker-1) | Pod Status after Creation request |
apiVersion: v1 name: my-pod imagePullPolicy: IfNotPresent resources: {} | IMAEGE IS NOT PRESENT ON THE NODE | IMAGE IS PRESENT ON THE NODE | kubectl get pod |
sudo crictl images |grep -i nginx | sudo crictl images |grep -i nginx | Events: |
Example-4: Using Always as an image pull policy
| POD-MANIFEST | Initial State of the image on the Scheduled node (Eg: kube-worker-1) | Image state after on the scheduled node after pod creation request(Eg: kube-worker-1) | Pod Status after Creation request |
apiVersion: v1 name: my-pod imagePullPolicy: Always resources: {} | IMAEGE IS PRESENT ON THE NODE | IMAGE IS PRESENT ON THE NODE | kubectl get pod |
sudo crictl images |grep -i nginx | sudo crictl images |grep -i nginx | Events: |
Pre-downloaded images vs downloading from the registry
| Using the image from the Registry | Using locally stored images |
| 1. It will be our responsibility to secure and manage the authentication to the registry. This is done via imagePullSecrets. 2. Downloading the image from the registry would consume some time(ms to minutes) 3. Users/admin does not(at least lesser management) need to manage the images on local nodes. If an image is missing, it will be downloaded. 4. Extremely suited for CI/CD environment. | 1. No need to manage the authentication to the registry. 2. Image downloading time is saved. 3. The worst part is managing the images locally, keeping them updated and secured. 4. Doing the cleanup and upload of the image need to be done manually. Not very compatible with CI/CD. |
The disadvantage of the above policies?
- The cluster Admin is not able to control what imagePullPolicy a rogue developer is using while creating the pod.
- The different developers may use different imagePullPolicy, bad for consistency.
Enforcing imagePullPolicy ?
To maintain the consistency in the cluster and ensure that every developer is pulling the image from the registry and not using the locally cached image we can enable the AlwaysPullImage admission plugin. This plugin will overwrite the ImagePullPolicy present in the request and mutate it to “Always” download the image before processing the request. You must have admin privileges to enable any admission plugin. Add the “AlwaysPullImages” to your list of admission plugins in the API Server manifest file.
vim /etc/kubernetes/manifests/kube-apiserver.yaml
- - --enable-admission-plugins=NodeRestriction,NamespaceAutoProvision
+ - --enable-admission-plugins=NodeRestriction,NamespaceAutoProvision,AlwaysPullImages| POD-MANIFEST | Initial State of the image on the Scheduled node (Eg: kube-worker-1) | Image state on the scheduled node after pod creation request(Eg: kube-worker-1) | Pod Status after Creation request |
apiVersion: v1 name: my-pod imagePullPolicy: Never resources: {} | IMAEGE IS NOT PRESENT ON THE NODE | IMAGE IS PRESENT ON THE NODE | kubectl get pod |
sudo crictl images |grep -i nginx | sudo crictl images |grep -i nginx | Events: |
You may also check the ImagePullPolicy of the POD, it will be set to always regardless of being set to Never in the manifest at the time of creation.
kubectl get pod my-pod -o jsonpath='{.spec.containers[0].imagePullPolicy}{"\n"}'
Always
Summary
Aggregate Rating
5 based on votes