Reconfigure VLR 9.1 appliance fails with error 'Failed to start ims-service: The connection was closed by the remote end during handshake. (1)'

Products

• VMware Live Recovery

Issue/Introduction

When you attempt to reconfigure a VMware Live Recovery (VLR) 9.1 appliance or establish site pairing, the operation fails. In the vSphere Client UI, the VMware Live Site Recovery plugin may still display an older version (such as 9.0) after an upgrade. Additionally, site pairing fails with a "Cannot connect to the other site" or "Connection not initiated" error.

ims-service is in 'Stopped' state. srm-server is in 'Started' state.

The following error is displayed in the UI during the appliance reconfiguration process: 

ERROR

Operation Failed

A general system error occurred: N7Vmacore15SystemExceptionE Failed to start ims-service: The connection was closed by the remote end during handshake. (1)

 

In the `vmware-dr.log` on the local appliance, you will observe log entries indicating authentication failures with the Remote STS (Secure Token Service) due to an expired password[cite: 1]:

DEBUG vmware-dr [SRM sub="Connection"] Attempting connection; 'vcsa.domain.local:443' ERROR vmware-dr [SRM sub="RemoteSite.RemoteStsServer.ConnHandler"] Unable to retrieve token from STS: --> N9SsoClient24PasswordExpiredExceptionE Authentication failed: Password of the user logging on is expired.:: Password of the user logging on is expired. User account expired: {Name: SRM-remote-xxxxxxxx, Domain: vsphere.local} WARNING vmware-dr [SRM sub="RemoteSite.RemoteStsServer" connID="sts-bm"] Failed to connect --> N9SsoClient24PasswordExpiredExceptionE Authentication failed: Password of the user logging on is expired.

Environment

Protection and Recovery 9.1, vCenter 9.1

Cause

The root cause of this issue is an expired `SRM-remote` solution service account password on the remote site vCenter Server[cite: 1]. This password expiration prevents the Secure Token Service (STS) authentication handshake from completing successfully. Because the connection to the remote STS fails, the new Inventory Mapping Service (`ims-service`) on VMware Live Recovery 9.1 cannot authenticate with the remote site and fails to start during reconfiguration.

Resolution

To resolve this issue, you must manually remove the expired solution account on the remote vCenter Server and re-establish the site pairing to force the regeneration of valid credentials.

Follow these steps to resolve the issue:

1. Log in via SSH to the remote vCenter Server Appliance (VCSA).

2. List the existing service accounts to verify their names by running the following command
   
   /usr/lib/vmware-vmafd/bin/dir-cli svcaccount list
   
3. Identify the expired `srm-remote-` account from the list

4. Delete the expired service account by running the following command (replace `srm-remote-xxxxxxxx` with your specific account name from step 2):
  
   /usr/lib/vmware-vmafd/bin/dir-cli svcaccount delete --name srm-remote-xxxxxxxx
   
5. Log in to the vSphere Client and access the Site Recovery UI.

6. Reconnect the site pair to force the regeneration of valid solution service account credentials.

7. Reconfigure the VMware Live Recovery appliance through its Virtual Appliance Management Interface (VAMI) page if necessary.

8. To prevent this issue from recurring, follow the detailed instructions in KB 367383 to set both the `SRM-` and `SRM-remote-` * solution accounts on each vCenter Server to "password never expires"

* "SRM-remote" if the two vCenters in site pair are Linked Groups in VCF.