Missing container images after getting a disk full

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:
vRA/vRO 8.0 pods will fail to start with a status of 'ErrImageNeverPull'. This can be checked with kubectl get pods --all-namespaces

Environment

VMware vRealize Orchestrator 8.x
VMware vRealize Automation 8.x

Cause

The version of K8s used in vRA/vRO 8.0 has a builtin image eviction mechanism that is triggered when a disk becomes full. There is no way to turn this mechanism off

Resolution

Reboot the affected vRA/vRO 8.0 appliance. There is a builtin recovery that will restore all images from a backup directory into the docker local storage so K8s can start the pods.