VCF 9.x Deployment Failure: Error LCMVROPSYSTEM25127 during Operations Collector Node Addition to the existing VCF Operations

Products

• VMware SDDC Manager
• VMware Cloud Foundation

Issue/Introduction

• VCF 9.x deployment fails at Join the existing operations appliance stage.
  
  
  
  
• Under VCF 9 deployment fails 'Retrieve the status of the VCF Operations collector node addition to the existing VCF Operations'. Throws an error message 'Request scaleout failed with error cause [{"messageId":"LCMVROPSYSTEM25127","message":"Error while configuring the adapter"
  
  
  
  
• Under SDDC Manager's /var/log/vmware/vcf/domainmanager/domainmanager.log below log snippets are found:
  
  YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,#####################,####] [c.v.e.s.v.s.GetRequestStatusWithRetryFromVcfOpsMgmtAction,dm-exec-#]  The request createenvironment failed, even after retry attempt with error cause [{"messageId":"LCMVROPSYSTEM25127","message":"Error while configuring the adapter.","eventId":"#####################","retry":true,"exceptionMessage":"Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [FQDN]","exceptionStackTrace":"com.vmware.vrealize.lcm.plugin.common.vrops.exceptions.AdapterConfigurationException: Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [<FQDN>]\n\tat com.vmware.vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.execute(VcfAdapterConfigurationTask.java:162)\n\tat com.vmware.vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.retry(VcfAdapterConfigurationTask.java:210)\n\tat com.vmware.vrealize.lcm.automata.core.TaskThread.run(TaskThread.java:60)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n","localizedMessageId":null,"parameters":null,"properties":{"skipTask":"false"}}]
  
  
  
• Under Fleet Manager's /var/log/vrlcm/vmware_vrlcm.log, below log snippets are found:
  
  YYYY-MM-DDTHH:MM:SS  INFO vrlcm[####] [pool-3-thread-##] [c.v.v.l.d.v.VropsEndpoint]  -- Response received. Status code: 400 Response data: {"type":"Error","message":"Invalid input format.","moreInformation":[{"name":"errorMessage"," value":"Failed to create VCF cloud account. vCenter instance id is missing for domain: ####"}],"httpStatusCode":400,"apiErrorCode":400}
YYYY-MM-DDTHH:MM:SS  ERROR vrlcm[####] [pool-3-thread-##] [c.v.v.l.p.c.v.t.VcfAdapterConfigurationTask]  -- Failed :: VMware Aria Operations Adapter Configuration
YYYY-MM-DDTHH:MM:SS  ERROR vrlcm[####] [pool-3-thread-91] [c.v.v.l.p.c.v.t.VcfAdapterConfigurationTask]  -- response data: {"type":"Error","message":"Invalid input format.","moreInformation":[{"name":"errorMessage","value":"Failed to create VCF cloud account. vCenter instance id is missing for domain: ####"}],"httpStatusCode":400,"apiErrorCode":400}
YYYY-MM-DDTHH:MM:SS  ERROR vrlcm[####] [pool-3-thread-91] [c.v.v.l.p.c.v.t.VcfAdapterConfigurationTask]  -- response message: 400
YYYY-MM-DDTHH:MM:SS  INFO vrlcm[####] [pool-3-thread-91] [c.v.v.l.p.a.s.Task]  -- Injecting task failure event. Error Code : 'LCMVROPSYSTEM25127', Retry : 'true', Causing Properties : '{ CAUSE :: skipTask ===  }' com.vmware.vrealize.lcm.plugin.common.vrops.exceptions.AdapterConfigurationException: Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [<FQDN>]
        at com.vmware.vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.execute(VcfAdapterConfigurationTask.java:162) [vmlcm-vropsplugin-core-9.x-SNAPSHOT.jar!/:?]
        at com.vmware.vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.retry(VcfAdapterConfigurationTask.java:210) [vmlcm-vropsplugin-core-9.x-SNAPSHOT.jar!/:?]
        at com.vmware.vrealize.lcm.automata.core.TaskThread.run(TaskThread.java:60) [vmlcm-engineservice-core-9.x-SNAPSHOT.jar!/:?]
  
  

Environment

• VMware Cloud Foundation 9.x
• SDDC Manger 9.x

Cause

The state of the Platform Services Controller (PSC), vCenter Server, or ESX host is not set to ACTIVE in the SDDC Manager PostgreSQL database.

Resolution

Contact Broadcom Support.