Deployment of VCF 9.0 through VCF installer fails with the error LCMVROPSYSTEM25127
search cancel

Deployment of VCF 9.0 through VCF installer fails with the error LCMVROPSYSTEM25127

book

Article ID: 422946

calendar_today

Updated On:

Products

VCF Operations

Issue/Introduction

  • Deployment of VCF 9.0 through VCF installer fails at Stage 12 VCF Adapter Configuration, the following extract is visible in the UI

    Request createenvironment failed with error cause [{"messageId":"LCMVROPSYSTEM25127","message":"Error while configuring the adapter.","eventId":"########-####-####-####-############","retry":true,"exceptionMessage":"Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [<SDDCmanagerFQDN>]","exceptionStackTrace":"com.vmware.vrealize.lcm.plugin.common.vrops.exceptions.AdapterConfigurationException: Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [<SDDCmanagerFQDN>]\n\tat com.vmware.vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.execute(VcfAdapterConfigurationTask.java:162)\n\tat com.vmware.vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.retry(VcfAdapterConfigurationTask.java:210)\n\tat com.vmware.vrealize.lcm.automata.core.TaskThread.run(TaskThread.java:60)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n","localizedMessageId":null,"parameters":null,"properties":{"skipTask":"false"}}] Reference Token: ######"

  • From /var/log/vmware/vcf/domainmanager/domainmanager.log on SDDC manager shows the task failing during a scale-out or configuration request

    YYYY-MM-DDTHH:MM:SS.Z+0000 ERROR [vcf_dm, 6983##a815d###2f1f4d###1596588be9, 407f] [c.v.e.s.v.s. GetRequestStatusWithRetryFromVcfOpsMgmtAction, dm-exec-35] Therequest scaleout failed, even after retry attempt with error cause [{"messageId": "LCMVROPSYSTEM25127", "message": "Error while configuring the adapter.", "eventId": "b4###7ff-7##2-4##0-a##0-d35######celd","retry": true, "exceptionMessage": "Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [<FQDN_OF_SDDC_MANAGER>]", "exceptionStackTrace": "com.vmware.vrealize.lcm.plugin.common.vrops.exceptions. AdapterConfigurationException: Some SDDC Manager hosts are not configured. Failed SDDC Manager hosts are: [<FQDN_OF_SDDC_MANAGER>] \n\tat com. vmware. vrealize. lcm. plugin. core. vrops. tasks. VcfAdapterConfigurationTask.execute (VcfAdapterConfigurationTask. java: 162) \n\tat com. vmware. vrealize.lcm.plugin.core.vrops.tasks.VcfAdapterConfigurationTask.retry (VcfAdapterConfiguratioTask.java: 210) \n\tat com.vmware.vrealize.lcm.automata.core.TaskThread. run (TaskThread. java: 60) \n\tat java.base/java.util. concurrent. ThreadPoolExecutor.runWorker (Unknown Source) \n\tat java.base/java.util. concurrent. ThreadPoolExecutor$Worker. run (Unknown Source) \n\tat java.base/java. lang. Thread. run (Unknown Source) \n""localizedMessageId": null, "parameters": null, "properties": { "skipTask": False"} } ]

  • The /storage/log/vcops/collector.log file on the Cloud Proxy reveals the root network failure:

    YYYY-MM-DDTHH:MM:SS.Z ERROR collector #### [ops@### threadId="#####" threadName="TasksManager-TaskHandler-###"] [com.integrien.alive.common.adapter3.CertificateChecker.checkCertificate] - Error trying to make connection
    java.net.NoRouteToHostException: No route to host"

    YYYY-MM-DDTHH:MM:SS.Z ERROR collector 8##6 [ops@4##3 threadId="###" threadName="TasksManager-TaskHandler-##"] [com.vmware.vcops.vlsi.VimClientFactory.createVimClientImpl] - Exception occurred while creating VIM client.
    com.vmware.vim.vmomi.client.exception.ConnectionException: https://<FQDN_OF_MANAGEMENT_vCENTER>/sdk invocation failed with "org.apache.http.conn.HttpHostConnectException: Connect to <FQDN_OF_MANAGEMENT_vCENTER>:443 [<FQDN_OF_MANAGEMENT_vCENTER>/##.##.###.##] failed: Connection refused"
         at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:265) ~[vlsi-client-8.0.2.jar :? ]
         at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.setResponseError(HttpExchangeBase.java:355) ~[vlsi-client-8.0.2.jar :? ]

    Caused by: org.apache.http.conn.HttpHostConnectException: Connect to <FQDN_OF_MANAGEMENT_vCENTER>:443 [<FQDN_OF_MANAGEMENT_vCENTER>/##.##.###.##] failed: Connection refused
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) ~[httpclient-4.5.jar:4.5]
    at org.apache.http.impl.conn. PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.jar:4.5]

  • Running a curl command on the Cloud Proxy to vCenter using the command curl -v telnet://<FQDN_OF_MANAGEMENT_vCENTER>:443 fails with the below error

    Host <FQDN_OF_MANAGEMENT_vCENTER>:443 was resolved.
    IPV6: (none)
    IPv4: ###.###.###.###
    Trying ###.###.###.###:443...
    connect to ###.###.###.### port 443 from ###.###.###.### port 35360 failed: No route to host
    Failed to connect to <FQDN_OF_MANAGEMENT_vCENTER> port 443 after 8142 ms: Could not connect to server
    closing connection #0
    curl: (7) Failed to connect to <FQDN_OF_MANAGEMENT_vCENTER> port 443 after 8142 ms: Could not connect to server

  • Running ping command on the Cloud Proxy to vCenter fails with the error Destination Host Unreachable 

Environment

VCF Operations 9.0.x

Cause

The primary cause is a network connectivity failure between the VCF Operations Cloud Proxy (Collector) and the Management vCenter Server.

For the VCF Adapter to configure correctly, the Cloud Proxy must have an unobstructed path to the vCenter over port 443. This issue typically occurs due to Physical or Virtual Firewall Blocks Traffic on port 443 is being dropped between the Cloud Proxy subnet and the Management network.

The Cloud Proxy is unable to find a valid route to the vCenter IP address.

Resolution

To resolve this issue, connectivity must be restored between the Cloud Proxy and the vCenter Server.

Step 1: Verify and Update Firewall Rules

Ensure that all firewalls between the Cloud Proxy and the Management vCenter have rules allowing the following:

  1. Source: Cloud Proxy IP

  2. Destination: Management vCenter IP

  3. Protocol/Port: TCP 443 (HTTPS/SDK)

Step 2: Validate Network Routing

  1. Log in to the Cloud Proxy via SSH.

  2. Check the routing table to ensure the gateway is correct: route -n.

  3. Ensure the Cloud Proxy and vCenter are on the VCF Management Network as defined during the Bring-up process.

Step 3: Migrate the Cloud Proxy (Workaround for Host-Specific Issues)

If the network configuration appears correct but connectivity still fails, there may be an issue with the specific ESXi host's networking:

  1. In the vSphere Client, locate the Cloud Proxy VM.

  2. Right-click the VM and select Migrate.

  3. Choose "Change compute resource only".

  4. Move the Cloud Proxy to a different ESXi host within the Management Cluster.

  5. Re-test connectivity using curl -v telnet://<vCenter_FQDN>:443.

Step 4: Retry the Deployment

Once the curl command returns a "Connected" status, return to the VCF Installer and click Retry on the failed task.

Powered by Wolken Software