Remove host from SDDC manager fails with error "Failed to fetch the cluster status from LCM for domain"

Products

• VMware SDDC Manager

Issue/Introduction

While removing host from SDDC manager, it fails with error "Failed to fetch the cluster status from LCM for domain <domain-ID>" on the GUI.

In commonsvcs logs, below entries can be noticed:

Error_type":"SERVICE_UNAVAILABLE","messages":[{"args":["
550","550","[email protected]"],"default_message":"User session count is limited to 550. Existing session count is 550 for user [email protected].","id":"com.vmware.vapi.endpoint.failedToLogin
MaxUserSessionCountReached"}]}}"

Environment

VMware Cloud Foundation 5.x

Cause

This issue happens due to the VAPI user session limit being reached on the vCenter server for user "[email protected]"

Resolution

Increase the session.maxSessionsPerUser value on the affected vCenter

1. Take appropriate snapshots of the vCenter(s)
2. SSH to the vCenter via root
3. Take a backup of /etc/vmware-vapi/endpoint.properties

cp /etc/vmware-vapi/endpoint.properties /etc/vmware-vapi/endpoint.properties.backup
   
   
4. Edit the endpoint.properties file
   
   vi /etc/vmware-vapi/endpoint.properties
   
   
5. Remove the comment on the session.maxSessionsPerUser and increase the value to 1000.  (Note: This parameter is not present in vCenter 8.0.X, need to manually add the set the value to 1000)
   
   #session.maxSessionCount=1000
session.maxSessionsPerUser=1000
# One hour is 3 600 000 millis
#session.maxSessionIdleMilliseconds=3600000
# One day is 86 400 000 millis
#session.minSessionLifespanMilliseconds=86400000
# Two days are 172 800 000 millis
#session.maxSessionLifespanMilliseconds=172800000
   
   
6. Restart vmware-vapi-endpoint service
   
   service-control --stop vmware-vapi-endpoint && service-control --start vmware-vapi-endpoint