Aria Automation pods failed to start with status "ImageInspectError" after executing deploy.sh

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Aria Automation is not accessible.
• When checking pods status with command "kubectl -n prelude get pod", some pods' status is "ImageInspectError"
• When checking pod's information with command "kubectl -n prelude describe pod <pod name>", there is "ERROR: readlink /var/lib/docker/overlay2: invalid argument"
• When running script "/opt/scripts/restore_docker_images.sh", it returns "Nothing to load"

Environment

Aria Automation 8.x

Resolution

Follow procedure below to restore Aria Automation service:

• Run command "/opt/scripts/deploy.sh --shutdown" on any Aria Automation node
• Power off all Aria Automation nodes
• Create snapshot for all Aria Automation nodes
• Power on all Aria Automation nodes
• Run command "kubectl get node" to make sure all nodes are "Ready"
• Run command "docker system prune -af" on all Aria Automation nodes one by one
• Run command "/opt/scripts/restore_docker_images.sh" on all Aria Automation nodes one by one
• Run command "/opt/scripts/deploy.sh" on any Aria Automation node