Error: "Unable to connect to vCenter" when creating a replication due to a stale manager service

Products

• VMware Live Recovery

Issue/Introduction

• In the System Monitoring view of the VMware Cloud Director Availability (VCDA) Replicator Service Management Portal, you see a stale offline Manager Service listed in the Manager Service status section.
• The related Manager Service displays an error stating:
  
  SSO Admin Username or Password are incorrect
  
  
• Creating a new outgoing replication fails with the error:

  Unable to connect to vCenter '#######-####-####-####-#########'

• On the local replicator within the log file /opt/vmware/h4/replicator/log/replicator.log the following is seen:
  
  WARN - [UI-<task>] [job-46] c.v.h.r.diagnostics.DiagnosticsService   : Unable to connect to vc <vc_uiud> with manager <manager_uuid> credentials
com.vmware.vlsi.client.exception.InvalidVcLoginException: (vim.fault.InvalidLogin) 
...
Caused by: com.vmware.vim.binding.vim.fault.InvalidLogin: Cannot complete login due to an incorrect user name or password.

Environment

• VMware Cloud Director Availability 4.x

Cause

This issue can occur when a Cloud Replication Management Appliance has been decommissioned or replaced where the Manager Service has not been unpaired with the local Replicator Service first.

Resolution

To resolve this issue perform the following:

1. Log in to the Replicator Appliance web interface.
2. Identify the offline instance ID by navigating to System Health.
3. Note down the offline Manager Instance ID which is visible in the Manager Service status section.
4. Verify if the offline Manager is a Local or Remote Manager. To do this, navigate to Manager Services and locate the offline Instance ID you just noted from step 3. Check its API URL — if the URL begins with https://tn-* instead of an IP address this confirms the instance belongs to a remote manager.
5. Confirm the instance is safe to remove by checking across all connected cloud sites. To do this, log in to the Manager service (using port 8044 or 8441) for all connected cloud sites. Compare their Manager Instance IDs against your offline ID. If you cannot find a match across any of the cloud sites, the offline instance is safe to remove.
6. To remove the offline service, return to the Replicator Appliance. Navigate to Manager Services, select the offline Manager Service ID, and click Remove.
7. Verify in the System Health dashboard on the Replicator Appliance that the offline manager is no longer displayed.
8. Perform a repair to to Peer Sites to ensure that all sites are informed that the stale Manager is removed as outlined in Repair with a remote site. Also repeat the repair process from the peer cloud site back to the primary site to ensure to process is performed bi-directional.

To work around this issue, contact Broadcom Support and note this Article ID (315158) in the problem description. For more information, see Creating and managing Broadcom support cases.