Rebuilt VCF ESXi hosts failing with "Upgrade failed, internal error" when attempting to upgrade via VCF Operations > Build > Lifecycle

Products

• VCF Operations
• VMware Cloud Foundation

Issue/Introduction

• VCF Operations UI > Build > Lifecycle > Management Domain > Upgrades attempted upgrade fails with errors 
  
  VCF LCM Service shows "ESX upgrade using vSphere Lifecycle Manager images failed. Error: Upgrade failed, internal error."


• Hosts were recently removed from the vCenter inventory and rebuilt/reimaged, then added back into the vCenter inventory.
• NTP service is not reconfigured on the ESXi hosts after the rebuild.
• Manual inventory sync from VCF Operations UI > Operate > Overview > Inventory > Management Domain > ACTIONS > Inventory Sync fails with the following error:
  
  Description Perform Brownfield Environment Validations
Progress Message Validation failed with the following errors: host <host_name>: NTP should be configured on the hosts, host <host_name>: NTP should be configured on the hosts
  
  

Environment

VCF 9.1.x
VCF Operations 9.1.x

Cause

NTP not being configured on the host is preventing a full sync from completing of the hosts that were rebuilt.

Resolution

1. Configure NTP on the ESX Hosts 
2. Re run the manual sync at VCF Operations UI > Operate > Overview > Inventory > Management Domain > ACTIONS > Inventory Sync.
3. Wait for the sync to complete.  This should only take a few minutes depending on the size of the environment.

The ESX upgrade should now proceed under VCF Operations UI > Build > Lifecycle > Management Domain > Upgrades.