Error: "Replication with id 'H4-########-####-####-####-########' was not found." when entering a Replicator Service into maintenance mode

Products

• VMware Cloud Director

Issue/Introduction

• When you try to remove a Replicator Service from a Manager Service via the VMware Cloud Director Availability (VCDA) Manager Service Admin Portal, you see an error similar to:
  
  Replicator Service '########-####-####-####-####e2c6' has active replications and can not be removed. Stop them and try again.
  
  
• When you put a Replicator Service into maintenance mode, the Enter maintenance mode task fails and you see an error similar to:
  
  Unable to switch Replicator Service for one or more replications.
  
  
• When a Replicator Service is put into maintenance mode, VCDA attempts to move the source replications to another Replicator in the same site. Instances of the Change source Replicator Service task fails with an error similar to:
  
  Replication with id 'H4-########-####-####-####-####029c' was not found.

Environment

VMware Cloud Director Availability 4.x

Cause

This issue occurs when a replication no longer exists on the source Replicator Service but is still present in the Cloud and Manager Services that map the stale replication to the Replicator Service. This blocks the ability to decommission the Replicator Service from the impacted site.

Resolution

To resolve this issue, identify the replications that are blocking the ability to enter the Replicator Service into maintenance mode:

1. Take note of the H4 IDs from each of the Change source Replicator Service task errors.
2. SSH to the Cloud Director Replication Management Appliance and log in as root.
3. Run the following command to retrieve more details about the stale replication:
   
   psql -U h4postgres -d h4cloud -c "SELECT manager_replication_id, id, computer_name, source_vm_moref from vmreplication where manager_replication_id='<H4 ID from UI error>';"
   
   
4. To verify you are experiencing this issue, log into the Replicator Service Admin Portal as root.
5. Using the output from step 3, verify the source VM MORef is not present in the Source Replications view.
6. Navigate to the VCDA Provider Portal and search for a replication with a name matching the Computer Name retrieved from step 3.
7. (Optional) If you need to keep the latest instance of the replication, failover the replication before proceeding to the next step.
8. Delete the stale replication from the VCDA Provider Portal.
9. Once the replication has been deleted, return to the Manger Service Admin Portal and remove the Replicator Service.