Commissioning Hosts in SDDC Manager Fails with Error "Failed to fetch VCF supported ESXi versions. Please retry after some time."

Products

• VMware SDDC Manager

Issue/Introduction

Commissioning Hosts in SDDC Manager Fails with Error "Failed to fetch VCF supported ESXi versions. Please retry after some time."

Operations Manager logs - /var/log/vmware/vcf/operationsmanager/operationsmanager.log

YYYY-MM-DDTHH:MM:SS  ERROR [vcf_om,83c1160036f44c54,d4b7] [c.v.v.h.f.v.s.u.HostFreePoolValidationUtils,host-comm-decomm-exec-1] Error while getting the VCF supported ESXi versions from LCM com.vmware.cloud.foundation.rest.lcm.runtime.ApiException:
at com.vmware.cloud.foundation.rest.lcm.runtime.ApiClient.handleResponse(ApiClient.java:926)
at com.vmware.cloud.foundation.rest.lcm.runtime.ApiClient.execute(ApiClient.java:842)
at com.vmware.cloud.foundation.rest.lcm.service.ImageManagementServiceApi.getCompliantVersionsForProductTypeWithHttpInfo(ImageManagementServiceApi.java:296)
at com.vmware.cloud.foundation.rest.lcm.service.ImageManagementServiceApi.getCompliantVersionsForProductType(ImageManagementServiceApi.java:279)
at com.vmware.vcf.hostmanager.freepool.validation.service.utils.HostFreePoolValidationUtils.getSupportedESXiVersions(HostFreePoolValidationUtils.java:95)
at com.vmware.vcf.hostmanager.freepool.validation.service.impl.VCFSupportedESXiVersionValidator.validateHosts(VCFSupportedESXiVersionValidator.java:80)
at com.vmware.vcf.hostmanager.comm.services.impl.CommissionHostsValidator.validateHosts(CommissionHostsValidator.java:213)
at com.vmware.vcf.hostmanager.comm.services.impl.CommissionHostsValidator.lambda$null$0(CommissionHostsValidator.java:149)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

 

lcm-debug.log - /var/log/vmware/vcf/lcm/lcm-debug.log

Caused by: java.lang.RuntimeException: MGMT domain collection size is not equal to 1
at com.vmware.evo.sddc.lcm.services.impl.ImageManagementServiceImpl.getVersionComplianceMatrixForDomainInternal(ImageManagementServiceImpl.java:464)... 112 common frames omitted
YYYY-MM-DDTHH:MM:SS ERROR [vcf_lcm,43d773a597b0496e,9007] [c.v.e.s.l.a.r.c.i.ImageManagementController,http-nio-127.0.0.1-7400-exec-8] Exception in getCompliantVersionsForProductType {} java.lang.RuntimeException: Exception while getting compliance versions for productType ESX

Environment

VMware Cloud Foundation 4.x

 

Cause

• Incorrect removal of hosts and clusters from SDDC DB. 
• Stale entries of clusters and hosts in the SDDC DB due to manual removal. 

Resolution

Attempt to remove the hosts using the internal API. Hosts APIs | VMware Cloud Foundation API

Please consult Broadcom Support to remove the hosts from the SDDC Manager DB. For more information, please see Creating and managing Broadcom support cases