Patching vCenter Server fails with error "Installation of one or more containers failed"

Products

• VMware vCenter Server
• VMware vCenter Server 8.0

Issue/Introduction

• vCenter Server update fails with error "Installation of one or more containers failed".
  
  
  
  
• Patching log file shows similar to below entries :
  

  /var/log/vmware/applmgmt/update_microservice.log

  YYYY-MM-DD HH:MM:MM,### - #### - update_functions_target::      runCommandAndCheckResult: 428 -    DEBUG - runCommandAndCheckResult failed: 'Traceback (most recent call last):\n  File "/usr/lib/containerfw/patch_containers.py", line 129, in <module>\n    patch(args.stage_dir)\n  File "/usr/lib/containerfw/patch_containers.py", line 70, in patch\n    remove_unused_images()\n  File "/usr/lib/containerfw/container_util/helper.py", line 194, in remove_unused_images\n    loaded_containers = [item.strip() for item in get_loaded_containers()]\n  File "/usr/lib/containerfw/container_util/helper.py", line 123, in get_loaded_containers\n    rc, text = run_command([CONTAINERD_CTR, \'container\', \'list\', \'-q\'])\n  File "/usr/lib/containerfw/container_util/helper.py", line 38, in run_command\n    raise Exception(\'Command failed: stdout %r stderr %r\' % (text, err))\nException: Command failed: stdout b\'\' stderr b\'ctr: failed to dial "/run/containerd/containerd.sock": context deadline exceeded: connection error: desc = "transport: error while dialing: dial unix:///run/containerd/containerd.sock: timeout"\\n\'\n'

  YYYY-MM-DD HH:MM:MM,### - #### -    update_b2b_target::                       install:2993 -    ERROR - Installation of Containers phase failed.

Cause

This issue is caused when "containerd" service fails to respond within the timeout window of 10 seconds during the patching process.

Resolution

This is a known issue with vCenter Server 8.x and Broadcom Engineering is actively working towards fixing this issue in a future release.

Workaround

Manually start the containerd service by following below steps and retry the patching :

1. Login to vCenter Server as 'root' account using SSH Client.
2. Start the 'containerd' service by executing below command:
   
   systemctl start containerd
   
   Note: This service will be by default in stopped state and patching process will attempt to start this service during vCenter Server update.
   
   
3. Resume vCenter update by clicking "Resume vCenter update" button in the failed patching window OR retry the patching if vCenter Server is already rolled back to the previous state.