Alert: "No connection to VR Server: Not responding" when replicating encrypted VM

Products

• VMware Cloud Director

Issue/Introduction

• On the VM for which the replication needs to be configured the below alert can been seen:
  
  No connection to VR Server: Not responding
  
  
• VM replication fails only when the VM is on a specific ESX.
• In the /opt/vmware/h4/manager/log on the destination site the below error is present:
  

  DATE TIME ERROR - [UI-########-####-####-########-zl-K3-YF] [sync-1] com.vmware.h4.jobengine.JobExecution     : Task ########-####-####-########7828 (WorkflowInfo{type='sync', resourceType='replication', resourceId='H4-########-####-####-########ed65
5a', isPrivate=false, resourceName='VM_NAME'}) has failed
com.vmware.h4.manager.api.exceptions.SyncTimeoutException: Sync timeout for replication 'H4-########-####-####-########655a'.
        at com.vmware.h4.manager.replication.sync.SyncManager.checkSyncHealthImpl(SyncManager.java:542)
        at com.vmware.h4.manager.replication.sync.SyncManager.checkSyncHealth(SyncManager.java:485)
        at com.vmware.h4.manager.replication.sync.SyncManager.lambda$startHealthMonitor$8(SyncManager.java:636)
        at com.vmware.h4.common.mdc.MDCRunnableWrapper.run(MDCRunnableWrapper.java:32)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

  
  
  
• In the /var/log/vmkernel.log of the affected ESXi the below entry is present:
  
  DATE TIME cpu127:51278132)WARNING: Hbr: 1110: Using vmknic vmk# on netstack defaultTcpipStack.
DATE TIME cpu125:51278352)WARNING: Hbr: 574: Connection failed to ##.##.##.## (groupID=H4--########-####-####-########655a): Connection refused
DATE TIME cpu125:51278352)WARNING: Hbr: 5093: Failed to establish connection to [##.##.##.##]:32032 (groupID=H4-########-####-####-########655a): Connection refused
  
  
• When migrating the VM to a different ESX the replication can be correctly configured.
• The VM is encrypted.
• The connectivity test from the affected ESX to the replicator appliance over the port 32032 (used for the replication for the encrypted VM) fails.
• The port 32032 is open on the replicator and the connectivity test from a not affected ESX succeeded.

Environment

VMware Cloud Director Availability 4.7.2

Cause

The  HBR agent VIB needed for replicating encrypted VM has not been installed on the ESX.

Resolution

Installing the HBR agent VIB as per documentation Create a replication for encrypted virtual machines