VCF 9.0.1 installation fails with "Populate SDDC manager inventory object and Credentials"
search cancel

VCF 9.0.1 installation fails with "Populate SDDC manager inventory object and Credentials"

book

Article ID: 414746

calendar_today

Updated On:

Products

VMware SDDC Manager

Issue/Introduction

  • The VCF 9.0.1 installation fails at "Populate SDDC manager inventory object and Credentials" as shown below: 

  • The domainmanager logs on vcf installer appliance at the path /var/log/vmware/vcf/domainmanger/domainmanager.log will report the following: 

YYYY-MM-DDTHH:MIN DEBUG [vcf_dm,####,7f8f] [c.v.v.s.c.s.SecurityConfigurationServiceImpl,dm-exec-33]  Security config retrieved {"fipsMode":true}
YYYY-MM-DDTHH:MIN ERROR [vcf_dm,####,7f8f] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-33]  [token id] VCF_GUEST_SERVICE_INVALID_HTTP_STATUS_CODE Invalid HTTP status code {0}, while configuring SDDC Manager inventory
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Invalid HTTP status code {0}, while configuring SDDC Manager inventory
        at com.vmware.evo.sddc.sddcmanager.PostInventoryAndCredentialsAction.execute(PostInventoryAndCredentialsAction.java:124)
        at com.vmware.evo.sddc.sddcmanager.PostInventoryAndCredentialsAction.execute(PostInventoryAndCredentialsAction.java:47)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:66)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:161)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:147)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:403)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:517)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:128)
        at jdk.internal.reflect.GeneratedMethodAccessor694.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:85)
        at com.google.common.eventbus.Subscriber.lambda$dispatchEvent$0(Subscriber.java:71)
        at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:63)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 : "{"errorCode":"INVENTORY_INTERNAL_SERVER_ERROR","arguments":[],"message":"Inventory Internal Server Error","referenceToken":"token id"}"
        at org.springframework.web.client.HttpServerErrorException.create(HttpServerErrorException.java:102)
        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:186)
        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:137)
        at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
        at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:942)
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:891)
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:790)
        at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:672)
        at com.vmware.evo.sddc.sddcmanager.PostInventoryAndCredentialsAction.execute(PostInventoryAndCredentialsAction.java:73)
        ... 16 common frames omitted
YYYY-MM-DDTHH:MIN DEBUG [vcf_dm,68adb6e284fe607a81f6a6c78316dba8,7f8f] [c.v.e.s.o.c.ProcessingTaskSubscriber,dm-exec-33]  Collected the following errors for task with name PostInventoryAndCredentialsAction and ID ####-####: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.evo.sddc.sddcmanager.messages)]]

  • /var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log list the ESX FQDN as already existing

YYYY-MM-DDTHH:MIN_SEC INFO  [common,####,554d] [c.v.e.s.i.s.EsxiInventoryServiceImpl,http-nio-127.0.0.1-7100-exec-2] Create Esxi - ####
YYYY-MM-DDTHH:MIN ERROR [common,####,554d] [c.v.e.s.i.s.EsxiInventoryServiceImpl,http-nio-127.0.0.1-7100-exec-2] ESXi <ESX FQDN> already exist: ##### <ESX FQDN> already exists
YYYY-MM-DDTHH:MIN ERROR [common,####,554d] [c.v.e.s.i.s.e.DomainExtensionsImpl,http-nio-127.0.0.1-7100-exec-2] Failed to post limited domain inventory com.vmware.evo.sddc.inventory.model.InventoryAlreadyExistsException: ####-####-#### <ESX FQDN> already exists

  • PostgreSQL database on the installer will report that all the hosts are active and healthy. 
    • SSH as VCF user to VCF Installer via Putty client. 
    • Switch to root using the 'su' command
    • Enter the PostgreSQL database using the command: psql -h localhost -U postgres -d platform
    • Execute platform=# select hostname, status from host; 

hostname    | status |
------------|--------|
 esx01.fqdn | ACTIVE |
 esx02.fqdn | ACTIVE |

 

Environment

VCF 9.0/VCF 9.0.1 

Cause

The original VCF 9.0 installation was discarded and replaced with a new VCF 9.0.1 deployment using the existing VCF 9.0 installer. The previous installation had successfully deployed vCenter but failed during subsequent configuration steps, which left the stale entries of the hosts in an ACTIVE state within the installer.

Resolution

To resolve the issue, deploy the appropriate VCF 9.0.1 installer and re-initiate the installation

 

Additional Information

Powered by Wolken Software