Error: No connection to VR Server for virtual machine on host in cluster : Unknown'. while configuring replication for VM
search cancel

Error: No connection to VR Server for virtual machine on host in cluster : Unknown'. while configuring replication for VM

book

Article ID: 387243

calendar_today

Updated On:

Products

VMware Live Recovery

Issue/Introduction

Symptoms :

  • All the connectivity between vCenter, SRM and vSphere replication servers look healthy and connected.

  • When replication is enabled, the task is finished and an error message appears on the user interface while the sync is happening.

    Error appears on vCenter UI for VM in replication is as below,



    Error - No connection to VR Server for virtual machine: Not responding

    Replication fails on SRM UI,

Environment

VMware vSphere Replication 8.x

VMware vSphere Replication 9.x

Cause

It is observed that the replication was failing with connectivity issues. "Failed to bind to any of the specified VMKs for connection to"

In this scenario, the management network is on 162.x.x.x series IP and dedicated network is 192.x.x.x series IP.

  • This can be verified on ESXi host logs under path /var/run/log/hbr-agent.log and the below events can be found.

2025-01-30T06:34:32.967Z In(166) hbr-agent-bin[3431566]: [0x0000002295ebd700] error: [ConfigManager] Failed to get config store object. Comp: esx, Grp: services, Key: hbrsvc_target_info, Id: 162.xx.xx.xx, Prop: certificate
2025-01-30T06:34:32.967Z In(166) hbr-agent-bin[3431566]: [0x0000002295ebd700] info: [ProxyConnection] Setting up secure tunnel to broker on 162.xx.xx.xx:32032
2025-01-30T06:34:32.967Z In(166) hbr-agent-bin[3431566]: [0x0000002295ebd700] info: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Bound to vmk: vmk1 for connection to 162.xx.xx.xx:32032
2025-01-30T06:34:32.967Z In(166) hbr-agent-bin[3431566]: [0x0000002295e3c700] error: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Failed to connect to 162.xx.xx.xx:32032. Using nic 'vmk1'. Error: Network is unreachable

2025-01-30T06:43:32.974Z In(166) hbr-agent-bin[3431566]: [0x0000002296080700] info: [ProxyConnection] Setting up secure tunnel to broker on 162.xx.xx.xx:32032
2025-01-30T06:43:32.974Z In(166) hbr-agent-bin[3431566]: [0x0000002296080700] info: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Bound to vmk: vmk1 for connection to 162.xx.xx.xx:32032
2025-01-30T06:43:32.974Z In(166) hbr-agent-bin[3431566]: [0x0000002295e3c700] error: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Failed to connect to 162.xx.xx.xx:32032. Using nic 'vmk1'. Error: Network is unreachable
2025-01-30T06:43:32.974Z In(166) hbr-agent-bin[3431566]: [0x0000002295e3c700] error: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Failed to bind to any of the specified VMKs for connection to 162.xx.xx.xx:32032
2025-01-30T06:43:32.974Z In(166) hbr-agent-bin[3431566]: [0x0000002295e3c700] error: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Failed to connect to broker on 162.xx.xx.xx:32032: Input/output error
2025-01-30T06:43:32.974Z In(166) hbr-agent-bin[3431566]: [0x0000002295e3c700] error: [Proxy [Group: ] -> [162.xx.xx.xx:32032]] Failed to connect to broker: Input/output error

        When there is a communication and connectivity issues between ESXi hosts and vSphere replication appliance which is used for replication network, the VM replication failures are expected.

  • If port 31031 is closed or blocked below error appears and VM replication fails.

YYYY-MM-DDT06:23:37.479Z Wa(180) vmkwarning: cpu31:22158964) WARNING: Hbr: 571: Connection failed to ######### (groupID=GID-e33fa3eb-d5f0-4d9f-9108-870eb2a4blf3) : Connection refused
YYYY-MM-DDT06:23:37.479Z Wa(180) vmkwarning: cpu31:22158964) WARNING: Hbr: 5362: Failed to establish connection to [##########] :31031 (groupID=GID-e33fa3eb-d5f0-4d9f-9108-870eb2a4blf3) : Connection refused
YYY-MM-DDT06:23:43.540Z Wa(180) vmkwarning: cpul:22157639) WARNING: Hbr: 571: Connection failed to ########## (groupID=GID-el4b5210-7087-4836-b0ff-ladcc11469c0) : Connection refused
YYYY-MM-DDT06:23:43.540Z Wa(180) vmkwarning: cpul:22157639) WARNING: Hbr: 5362: Failed to establish connection to [##########] :31031 (groupID=GID-e14b5210-7087-4836-b0ff-ladcc11469c0) : Connection refused

  • Ping Fails for port 31031

    [root@r650-9014 :~ ] nc -zv ######### 31031
    nc: connect to ######### port 31031 (tcp) failed: Connection refused

 

Resolution

  • In vSphere Replication, initial and continuous replication traffic is handled on port 31031. VMs in replication fail because of a communication breakdown between the source hosts and the destination VR on port 31031

  • When there is a dedicated network configured for vSphere replication from both ESXi hosts and vSphere replication appliance, we need to configure the "IP Address for Incoming Storage Traffic" to enable the communication to work with defined/dedicated network. Refer the below link for instructions.

Create a VM Network Adapter to Use for Incoming Replication Traffic on the Combined vSphere Replication Appliance

When there is a dedicated network configured for replication purpose and this network not to be interrupted from management network, it is required to set the "IP Address for Incoming Storage Traffic" on appliance to address the issue.

After making the change we see the traffic established using dedicated network adapter and sync started successfully.

  • This can be verified on ESXi host logs under path /var/run/log/hbr-agent.log and the below events can be found.

2025-01-30T06:43:38.474Z In(166) hbr-agent-bin[3431566]: [0x0000002295ebd700] info: [Proxy [Group: GID-9d365f20-1808-47ec-881d-92373b3451c4] -> [192.xx.xx.xx:32032]] Setting up secure tunnel to server on 192.xx.xx.xx:32032
2025-01-30T06:43:38.474Z In(166) hbr-agent-bin[3431566]: [0x0000002295ebd700] info: [Proxy [Group: GID-9d365f20-1808-47ec-881d-92373b3451c4] -> [192.xx.xx.xx:32032]] Bound to vmk: vmk1 for connection to 192.xx.xx.xx:32032

Powered by Wolken Software