No connection to VR Server: Unknown

Products

• VMware vSphere ESXi

Issue/Introduction

• Virtual machine with replication configured shows the below alert in vCenter server:

• When trying to configure replication for a VM using vSphere replication following error is encountered 

Error : A replication error occurred at the vSphere Replication Server for replication "VM name: test". Details: 'No connection to VR Server for virtual machine "VM name: test" on host "ESXi host : host1" in cluster "Cluster name : cluster1" in "Cluster name : cluster1" : Unknown'.

• Virtual Machines may be showing as "Not Active (RPO Violation)" on some ESXI hosts but may be working when vMotioned to other ESXI hosts. 

Environment

VMware vSphere Replication 8.x
VMware vSphere Replication 9.x

Cause

This issue occurs when the target vSphere Replication appliance is unable to communicate with the source ESXi host's VMkernel interface, with replication traffic tagged, due to a missing static route, especially when the source is on a different subnet.

Resolution

If the appliance can't reach the source OR destination network (it depends where the appliance is placed), you need to add a static route. To add a static route to the vSphere Replication, you need to SSH the appliance. By default SSH service is disabled and you need to enable SSH on the VRA

• Access the vSphere Replication's console via HTML client & login as root
• From the console, run the following command to enable the SSH service in the appliance:

root@V####R [ ~ ]# /usr/bin/enable-sshd.sh

root@V####R[ ~ ]# service sshd start

• Add the route details in the file below to make the change persistent across reboots.

In the above screenshot: 

10-eth0.network -> Management

10-eth1.network  -> VR Traffic

10-eth2.network  -> VR NFC Traffic

 

NOTE: Edit the NIC configured for VR traffic, this could depend on your NIC configuration in the appliance. Below is an example from the KB: Isolating the Network Traffic of vSphere Replication

VRMSPR is the prod site vSphere replication appliance

VRMSDR is the Recovery site vSphere replication appliance

In the above screenshot, a static route entry for the source ESXi host network (192.168.50.1/24) has been added to the 10-eth1.network file on the DR site replication appliance, which is used by a VMkernel interface to handle replication traffic.

In the above screenshot, a static route entry for the source ESXi host network (192.30.50.1/24) has been added to the 10-eth1.network file on the PROD site replication appliance, which is used by a VMkernel interface to handle replication traffic.

 

Running tests to check if static route is working:

• To check if the configuration has been applied in the routing table of vSphere replication appliance, run the command:

# netstat –r

• To verify the VR appliance & hosts are reachable from the source & target hosts bi-directionally, run the vmkping command
• SSH into the source & target host and run this command to check if the hosts are able to reach the vSphere Replication appliance:

vmkping -I <VMK# of VR network> <IP address of VR appliance>

You can also PING the hosts from the VR appliance to test connectivity.

Static routes are specifically required when the host must transport traffic to a destination network outside of it's own datacenter that has a different IP range. 

 

Additional Information

For detailed information on how traffic is isolated in vSphere replication refer to this document: Isolating the Network Traffic of vSphere Replication