After restarting Docker service on one of the nodes, services are unhealthy, and deployments are failing

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:
User is unable to perform anything on the environment due to Internal Server Errors from most of the services






Check if the docker service has been restarted by executing the following command on the environment:

service docker status

and check how long has the service been up.



Active: 1 day 4 h ago

Environment

VMware vRealize Automation 8.x

Cause

Restarting of the Docker service on any of the nodes. Cause of the restart: manual restart of the service.
              

CLI commands:

               - service docker stop -> service docker start
               - service docker restart

Resolution

Two ways to recover from such crash of services after docker service restart:

• Redeploy of services(which fixes the problem immediately):

              execute the "/opt/scripts/deploy.sh" script without any parameters, just the deploy.sh

• Reboot the affected node and wait some time for things to go into normal state again(about 30-50 mins)