Reconnecting Fleet Management with VCF operation fails with error "Internal server error"
search cancel

Reconnecting Fleet Management with VCF operation fails with error "Internal server error"

book

Article ID: 430561

calendar_today

Updated On:

Products

VCF Operations

Issue/Introduction

  • Previously disconnected Fleet Management with VCF operations 
  • Reconnecting Fleet Management fails with "Internal server error" but applying KB Troubleshooting VCF Operations fleet management node Integration Failures in the VCF Operations allows you to reconnect Fleet Management with VCF operations 
  • Operations component in Fleet Management shows "Deployment Failed" after reconnecting Fleet Management with VCF operations as a result cannot manage the Operations component
  • There are no tasks indicating there was a previously failed deployment. 
  • VCF Operations for Networks/VCF Operations for Logs is deployed successfully in the environment
  • Triggering inventory sync for VCF operations for Networks fails with the following error in the UI:
    LCMVRNICONFIG90151
    Error Code: LCMVRNICONFIG90151
    LCMVRNICONFIG90151
    Error occurred while pushing capabilities to VCF Ops Networks
  • Similarly triggering inventory sync for VCF Operations for Logs fails with the following error in the UI: 
    Error Code: LCMVRLICONFIG40119
    LCMVRLICONFIG40119
    Error occurred while pushing capabilities to VCF Ops Logs,
    statusCode: 500: ("errorMessage":"Could not save
    configuration"}:{"errorMessage":"Could not save
    configuration"}
  • In VCF operations the /storage/log/vcops/log/web-vcf.log shows the error 
    2026-01-27T10:25:33.809Z ERROR web-vcf 1597 [ops@4413 threadId="164" threadName="Thread-8"] [com.vmware.vcf.operations.ui.data.DataProvider.error] - Connection reset
    java.net.SocketException: Connection reset
    
    2026-01-27T08:58:39.291Z ERROR web-vcf 1597 [ops@4413 threadId="302" threadName="ajp-nio-127.0.0.1-8009-exec-28"] [com.vmware.vcf.operations.ui.proxy.ProxyUtil.error] - <fleetmanagementfqdn>: Temporary failure in name resolution
  • nslookup and reverse lookup of fleet management, VCF operations and vCenter succeeds

Environment

VCF operations 9.x

VCF Fleet Management 9.x

Cause

There was an issue with DNS when registration with VCF operations and Fleet Management initially was done. The VCF Operations - Fleet Management integration requires the FQDN to register and communicate with dependent components. Upon triaging the log bundle, the /casa/cluster/status API on VCF Operations returns the IP address of the master node instead of the FQDN in the address field. When Fleet Management attempts to register VCF Operations, it relies on this API to obtain the FQDN. Since the API returned an IP address instead, Fleet Management could not complete the registration process.

Resolution

Note: Prior to applying the below steps please ensure you take snapshots for :

  1. Disconnect Fleet Management integration with VCF operations
  2. Enable Fleet Management UI by opening ssh session to Fleet Management as root and running the command 
    touch /var/lib/vrlcm/UI_ENABLED
  3. Open Fleet Management UI by entering https://<fleetmanagementfqdn> in the browser
  4. Navigate to Lifecycle → Components → Locate the VCF operations for Logs component → Click on 3 dots → Click on Delete → Ensure you do not select to delete the VM from vCenter
  5. Navigate to Lifecycle → Components → Locate the VCF operations for Networks component → Click on 3 dots → Click on Delete → Ensure you do not select to delete the VM from vCenter
  6. Reconnect Fleet Management integration with VCF Operations you may need to reapply KB in order to do so Troubleshooting VCF Operations fleet management node Integration Failures in the VCF Operations
  7. Open Fleet Management UI  → Lifecycle → VCF Management → Components → Click on Add Component → Select VCF operations for Logs → Choose  Installation Type as Import →  Click Next and provide the required details → Complete the import process.
  8. Repeat step 7 to import VCF Operations for Network 
  9. Disable Fleet Management UI by opening ssh session to Fleet Management as root and running the command : 
    rm /var/lib/vrlcm/UI_ENABLED