"There is no network connectivity between the source host and the target" after VLR upgrade
Article ID: 414263
Updated On:
Products
Issue/Introduction
After upgrade to VLSR 9.0.4, or VLR9.0.5, we are not able to communicate between source VR appliance and target ESXi hosts.
There is no network connectivity between the source host (id: 'host-#####', name: '#####') and the target broker 'Replication_appliance_IP_address'. Details: 'Connect: Input/output error'.
The source host (id: 'host-####', name: 'esxi_host_fqdn') successfully connected to the target broker 'Replication_appliance_IP_address', but there is no network connectivity between the source host 'esxi_host_fqdn' and the target host (id: 'host-####', name: '#######'). Details: 'Connect: Input/output error'.
On ESX /var/log/hbr-agent.log
... Failed to bind to any of the specified VMKs for connection to ##.##.##.##:32032
... Failed to connect to broker on ##.##.##.##:32032: Input/output error
On individual Replication view error:
A replication error occurred at the vSphere Replication Server for replication '######'. Details: 'No connection to VR Server for virtual machine ###### on host #### in cluster ### in ##: Unknown'.
On VLR appliance Enhanced Replication Mappings UI error and in var/log/vmware/hms/hms.##.log
There is no network connectivity between the source host (id: 'host-####', name: '##############') and the target broker '##.##.##.##'. Details: '401 Unauthorized from GET https://##.##.##.##/hbragent/api/v1.1/appPing.
VLR /var/log/vmware/hms/hms.###.log INFO response.filter.I18nActivationResponseFilter [tcweb-12] (..response.filter.I18nActivationResponseFilter) {operationID=....-HMS-5381} [] | The localized message is: There is no network connectivity between the source host (id: 'host-####', name: '#########') and the target broker '##.##.##.##'. Details: 'Connect: Input/output error'.
Environment
VMware Live Recovery 9.0.4
VMware Live Recovery 9.0.5
Cause
By default, the combined VMware Live Recovery appliance has one VM network adapter.
With enhanced replication mode, the VR broker service in target VLR appliance needs to talk to source ESX server's and vice-versa.
Even though the broker does not accept replication traffic, it still needs to talk with source ESX's replication network. The ESX hosts are configured to isolate replication traffic to a separate network. However, the VLR broker service listens on the management network IP. Because the source ESX replication network could not route to the VLR management network, the handshake failed.
Resolution
If the ESXi replication network is isolated from the management network, it will be unable to connect to the broker service.
There are 2 options available.
- Configure/Tag the same network services on the ESXi management network.
Perform this procedure for every ESX host that is used as a replication source.
1) In the vSphere Client, navigate to the ESX host.
2) Click the Configure tab and under Networking, select VMkernel adapters.
3) Edit the settings of the management network
By default the vmk0 adapter is used for management traffic.
4) Under Available services, select vSphere Replication and vSphere Replication NFC services for the local and remote ESXi replication network adapter.
5) Apply the settings and finish to complete the wizard.
Re-run the Replication Mapping test and it should succeed.
- Add a 2nd NIC to the VLR appliance See Techdoc Create a VM Network Adapter to Use for Incoming Replication Traffic on the Combined
VMware Live Recovery Appliance
Additional Information
Note :
If ESX needs to isolate replication traffic to a separate network, it will be unable to connect to VLR broker service as it is listening management network IP.
As VLR config UI does not support setting additional network for replication, we need perform these steps to manually edit the /etc/systemd/network/10-eth1.network
Techdoc: Configure a Static Route on an Additional VM Network Adapter
Feedback
