Enhanced Replication Mappings report error : "Fault occurred while performing health check. Details: 'Connect: Input/output error'."

Products

• VMware Live Recovery

Issue/Introduction

Symptoms:

• Enhanced Replication Mapping in Site Recovery UI reports following error:
  "Fault occurred while performing health check. Details: 'Connect: Input/output error'."
  
  
  
  
• Enhanced replication fails for VMs failed with
  "A replication error occurred at the vSphere Replication Server for replication '###'. Details: 'No connection to VR Server for virtual machine ### on host ### in cluster ### in ###: Unknown'"

Environment

VMware vSphere Replication 8.x

VMware vSphere Replication 9.x

Cause

• Either multiple vmkernel adapters on target ESXi hosts is enabled with the vSphere Replication and vSphere Replication NFC services or incorrect vmkernel adapter is enabled with these services
  
  
• hbr-agent.log (on source ESXi host) reports Input/output error while trying to connect to a target ESXi vmkernel adapter over port 32032:
  
  

(in /var/run/log/hbr-agent.log)

2025-01-23T08:05:18.907Z In(166) hbr-agent-bin[2172222]: [0x000000e886f80700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#6:32032]] Failed to connect to server ##.##.##.#6:32032 using broker info: Input/output error
2025-01-23T08:05:18.908Z In(166) hbr-agent-bin[2172222]: [0x000000e886cf1700] info: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#6:32032]] Setting up secure tunnel to brokered server ##.##.##.#6:32032 (2 of 2)
2025-01-23T08:05:18.908Z In(166) hbr-agent-bin[2172222]: [0x000000e886cf1700] info: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#2:32032]] Bound to vmk: vmk## for connection to ##.##.##.#6:32032
2025-01-23T08:06:06.036Z In(166) hbr-agent-bin[2172222]: [0x000000e886f80700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#3:32032]] Failed to connect to ##.##.##.#3:32032. Using nic 'vmk##'. Error: Connection timed out
2025-01-23T08:06:06.036Z In(166) hbr-agent-bin[2172222]: [0x000000e886f80700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#3:32032]] Failed to bind to any of the specified VMKs for connection to ##.##.##.#3:32032
2025-01-23T08:06:06.036Z In(166) hbr-agent-bin[2172222]: [0x000000e886f80700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#3:32032]] Failed to connect to server ##.##.##.#3:32032 using broker info: Input/output error
2025-01-23T08:06:06.037Z In(166) hbr-agent-bin[2172222]: [0x000000e886f80700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#3:32032]] Exhausted all server endpoints reported by broker.
2025-01-23T08:06:33.916Z In(166) hbr-agent-bin[2172222]: [0x000000e886d72700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#2:32032]] Failed to connect to ##.##.##.#2:32032. Using nic 'vmk##'. Error: Connection timed out
2025-01-23T08:06:33.916Z In(166) hbr-agent-bin[2172222]: [0x000000e886d72700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#2:32032]] Failed to bind to any of the specified VMKs for connection to ##.##.##.#2:32032
2025-01-23T08:06:33.916Z In(166) hbr-agent-bin[2172222]: [0x000000e886d72700] error: [Proxy [Group: GID-#######-a742-####-8f01-#######a3a8b] -> [##.##.##.#2:32032]] Failed to connect to server ##.##.##.#2:32032 using broker info: Input/output error


Resolution

• Enable 'vSphere Replication' and 'vSphere Replication NFC' traffic on vmkernel adapter of Target ESXi to which Source ESXi host is allowing the connection over port 32032.

(Validate it by running command on Source ESXi : nc -vz <Target-ESXi-vmk-IP> 32032)

(Note : For multiple vmkernel adapters enabled with these two traffic types (i.e. vSphere Replication & vSphere Replication NFC), disable on the vmk adapter where the these traffic is not allowed).

• Run the 'Run Tests' under Enhanced Replication Mapping to validate the results.