VCSA upgrade fails with error "internal error occurs during VMware vSphere Update Manager pre-upgrade checks"

Products

• VMware vCenter Server

Issue/Introduction

• VCSA upgrade fails during pre-upgrade checks at stage 2 with "internal error occurs during VMware vSphere Update Manager pre-upgrade checks"

  
  
  
  
  

• In the CollectRequirements_com.vmware.vcIntegrity_XXXX_XX_XX_XX_XX.log located under /var/log/vmware/upgrade/ you will see these entries. 
  
  

  stderr: 
  YYYY-MM-DDTHH:MM:SS<Time_Zone> INFO service_manager Service vmware-ufad-vci reported status stopped. Expected status started
YYYY-MM-DDTHH:MM:SS<Time_Zone>  ERROR service_manager Service cannot be started. Error: Failed to start vmware-ufad-vci.service: Unit vmware-ufad-vci.service failed to load: No such file or directory.
YYYY-MM-DDTHH:MM:SS<Time_Zone>  ERROR __main__ Upgrade Phase 'vcIntegrity:CollectRequirements' failed. Exception: Service cannot be started. Error: Failed to start vmware-ufad-vci.service: Unit vmware-ufad-vci.service failed to load: No such file or directory.

  Traceback (most recent call last):
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/payload/componentPhaseLauncher.py", line 461, in main  executionResult = systemExtension(exeContext)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/libs/sdk/extensions.py", line 94, in __call__result = self.extension(*args)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/libs/sdk/extensions.py", line 110, in _func return func(*args)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/payload/component-scripts/vcIntegrity/__init__.py", line 40, in do CollectRequirementsreturn vcIntegrity_upgrade.collectRequirements(context)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/payload/component-scripts/vcIntegrity/vcIntegrity_upgrade.py", line 369, in collectRequirements return collectRequirementsLin(context)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/payload/component-scripts/vcIntegrity/vcIntegrity_upgrade.py", line 271, in collectRequirementsLin serviceManager.start(VUM_SERVICE_NAME)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/libs/sdk/service_manager.py", line 979, in wrapper return getattr(controller, attr)(*args, **kwargs)
 File "/tmp/vmware-upgrade-temp-dirP1UlVsK204/tmpXhCR8ilrFq/libs/sdk/service_manager.py", line 745, in start raise IllegalServiceOperation(errorText)
service_manager.IllegalServiceOperation: Service cannot be started. Error: Failed to start vmware-ufad-vci.service: Unit vmware-ufad-vci.service failed to load: No such file or directory.

Environment

VMware vCenter Server 8.0.x

VMware vCenter Server 7.0.x
VMware vCenter Server 6.7.x

Cause

• This is caused if the update manager service in the source vCenter is stopped when the upgrade was performed. 
• The 100% disk utilization of /storage/log causes the service start failure. 

 

Resolution

• Check the status of update manager service.
• service-control --status --all or service-control --status updatemgr
• If it is not running try to start the service and troubleshoot in case of any errors. 
• Update manager service may go in stopped state even if started with command:
• service-control --start vmware-updatemgr 
• Investigate /storage/log partition using below kb article:
  vCenter log disk exhaustion or /storage/log full

Additional Information

• vCenter upgrade will fail at stage 2 pre-upgrade check