Reconnect fails between VMware Live Recovery sites

Products

• VMware Live Recovery

Issue/Introduction

Symptoms: 

1. Reconfigure in the VAMI interface is successful on both sides.
2. Reconnect fails with com.vmware.vapi.std.errors.already_exists during the createSolutionUser operations

 

The UI will show the following error:
A general system error occurred: N2Dr13VapiExceptionE Error: com.vmware.vapi.std.errors.already_exists
Messages:

com.vmware.vcenter.svcaccountmgmt.alreadyexists<Service account for given name already exists (Entry already exists, VMware directory error)

 

On the Live Recovery Appliance, vmware-dr.log will show messages like this (check both sides): 

DEBUG vmware-dr 795380 [SRM@4413 sub="LocalSite.LocalSaMgmtServer" opID="dd882f9d-8fc1-4f94-8098-###########-createSolutionUser"] Starting create service account 'SRM-remote-########-####-####-####-###########', overwrite='1'
DEBUG vmware-dr 796110 [SRM@4413 sub="LocalSite.LocalSaMgmtServer" opID="dd882f9d-8fc1-4f94-8098-###########-createSolutionUser"] Try to delete service account: ignoring delete error:
--> N2Dr13VapiExceptionE Error:
-->    com.vmware.vapi.std.errors.internal_server_error
--> Messages:
-->    vapi.bindings.method.impl.unexpected<Provider method implementation threw unexpected exception: com.vmware.vapi.std.errors.Unauthorized>
-->    com.vmware.vcenter.svcaccountmgmt.unauthorized<Failed to delete Service Account and it's permissions (Failed to delete permissions of user SRM-remote-########-####-####-####-###########)>
-->
--> 
ERROR vmware-dr 796015 [SRM@4413 sub="DrSiteUserManager" opID="dd882f9d-8fc1-4f94-8098-###########-createSolutionUser"] Failed to create service account 'SRM-remote-########-####-####-####-###########' for SRM server '########-####-####-####-###########':
--> N2Dr13VapiExceptionE Error:
-->    com.vmware.vapi.std.errors.already_exists
--> Messages:
-->    com.vmware.vcenter.svcaccountmgmt.alreadyexists<Service account for given name already exists (Entry already exists, VMware directory error[9706])>

 

On the vCenter server, /var/log/vmware/sso/svcaccountmgmt.log will show messages like this:

INFO vmidentity-svcaccountmgmt 52 [vc@4413 threadName="tomcat-http--11" logger="com.vmware.vcenter.svcaccountmgmt.impl.ServiceAccount" corId="997b303d-51d4-4722-847e-###########"] Creating Service Account : SRM-remote-########-####-####-####-###########
ERROR vmidentity-svcaccountmgmt 52 [vc@4413 threadName="tomcat-http--11" logger="com.vmware.vcenter.svcaccountmgmt.impl.ServiceAccount" corId="997b303d-51d4-4722-847e-###########"] createServiceAccount: Got Directory Exception

Environment

VMware Live Recovery

Protection and Recovery

Cause

A stale solution user account exists in the vCenter configuration that prevents the SRM appliance from provisioning a new service account.

Resolution

Identification and Removal Steps

1. Identify the Conflicting Solution User Account The conflicting account is logged in the vmware-dr (Site Recovery Manager) logs.

• Log Location:
  /var/log/vmware/dr/vmware-dr.log (on the SRM appliance)
  /var/log/vmware/sso/svcaccountmgmt.log (on the vCenter appliance)
  
  
• Search String: Search for "create service account" or the error message: com.vmware.vapi.std.errors.already_exists.
  
  
• Identify the User ID: The log will output a specific entry format, such as: 
  Starting create service account 'SRM-remote-#########-####-####-####-############' 
  
  
• Copy the full string SRM-remote-[UUID] exactly as it appears in the logs. This is the identifier you must remove.

2. Remove the Stale Solution User Account 
Note: This must be performed in the vCenter Server environment where the SRM/VLR solution is registered.

• SSH to the vCenter Server Appliance

• Run the following commands to list the service accounts on the vCenter. Validate that the remote SRM account which were found in the svcaccountmgmt.log and vmware-dr.log are listed here:

  • • /usr/lib/vmware-vmafd/bin/dir-cli svcaccount list

• Then proceed to delete the remote SRM service account identified in the logs using the command:

  • • /usr/lib/vmware-vmafd/bin/dir-cli svcaccount delete --name <SRM-remote-########-####-####-####-############>

3. Restore Site Connectivity

1. 1. Return to the VMware Live Recovery (VLR) / SRM interface.
      
      
   2. Navigate to the site pairing configuration.
      
      
   3. Select the disconnected site and click Reconnect.
      
      
   4. The system will now successfully provision a new, clean service account and re-establish the connection.