Pairing fails with the error: "Unable to connect to Lookup Service at https://target_vc:443/lookupservice/sdk. Reason: invocation failed with java.net.NoRouteToHostException: No route to host" after converging/upgrading to vLR 9.0.5

Products

• VMware Live Recovery

Issue/Introduction

Symptoms:

• Site pairing fails after converge/upgrade to vLR 9.0.5.0.
• Connectivity from Source_VLR to target_vc on port 443 fails with error “No route to host”

root@source_vlr [ /var/log/vmware/hms ]# curl -v telnet://target_vc:443
* Host target_vc:443 was resolved.
* IPv6: (none)
* IPv4: 172.17.##.##
*   Trying 172.17.##.##:443...
* connect to 172.17.##.## port 443 from 172.17.##.## port 36594 failed: No route to host
* Failed to connect to target_vc port 443 after 3056 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to target_vc port 443 after 3056 ms: Could not connect to server

• Ping to target_vc fails with Destination Host Unreachable

root@source_vlr [ /etc/systemd/network ]# ping target_vc_ip
PING 172.17.##.### (172.17.##.###) 56(84) bytes of data.
From 172.17.##.### icmp_seq=1 Destination Host Unreachable
ping: sendmsg: No route to host
From 172.17.##.### icmp_seq=23 Destination Host Unreachable

--- 172.17.##.### ping statistics ---
102 packets transmitted, 0 received, +99 errors, 100% packet loss, time 103400ms

• From Source_VLR, communication to local vCenter (Source_VC) works fine

Environment

• VMware Live Recovery 9.0.4
• VMware Live Recovery 9.0.5

Cause

The issue is caused by a network conflict within the VLR appliance due to the Docker bridge network (172.17.#.#/16) overlapping with the target site network (172.17.##.0/24).

In Linux, routing decisions are based on the destination subnet.

Because of this overlap:

• Traffic from AH to BH is incorrectly routed through the Docker bridge interface instead of the main network interface
• ARP resolution fails (entries showing as <incomplete>)
• This results in “No route to host” errors on port 443

Validating the Cause:

• Run the docker network command to get the Network ID for snapservice_default

root@AHPPVLR01 [ ~ ]# docker network ls
NETWORK ID     NAME                  DRIVER    SCOPE
234030941a5e   bridge                bridge    local
4861848feaac   host                  host      local
eca3ef7fc5c6   none                  null      local
9a3cb6593d00   snapservice_default   bridge    local

• Run the "docker network inspect" command to get the Subnet details.

root@source_vr [ ~ ]# docker network inspect 9a3cb6593d00
[
    {
        "Name": "snapservice_default",
        "Id": "9a3cb6593d00c52a3689e5560aaadec3e3########538c36db92854eb649",
           "Config": [
                {
                    "Subnet": "172.17.#.#/16",
                    "Gateway": "172.17.#.#"
                }
            ]

• From the ARP table:

root@source_vr [ /etc/systemd/network ]# arp -a
target_vr (172.17.##.###) at <incomplete> on br-9a3cb6593d00
? (172.17.##.#) at <incomplete> on br-9a3cb6593d00
target_vc (172.17.##.###) at <incomplete> on br-9a3cb6593d00
source_vc (172.16.##.###) at 00:0c:29:##:##:## [ether] on eth0
_gateway (172.16.##.#) at 00:09:0f:##:##:## [ether] on eth0

• All target site IPs are resolving via the Docker bridge interface
• ARP entries remain incomplete, meaning MAC address cannot be resolved
• At the same time, local network entries on eth0 work as expected

This confirms that traffic is incorrectly routed through the Docker bridge network.

Resolution

• Avoid using Docker networks that overlap with production or remote site networks
• Change the Docker bridge network to a non-overlapping subnet (for example: 169.254.x.x or 192.168.x.x)
• Steps to change subnet for snapservice_default

1. Take a snapshot of the VLR appliance.

2. Remove the existing network

# docker network rm snapservice_default

3. Recreate with a non-overlapping subnet - Choose a subnet that does not overlap (example: 192.168.100.0/24 or 169.254.0.0/16)

# docker network create --driver bridge --ipv6 --subnet 192.17.#.#/16 --gateway 192.17.##.## --subnet 2001:db8:1::/64 snapservice_default

4. Verify

# docker network inspect snapservice_default

5. Restart services:

# systemctl restart docker

# systemctl restart systemd-networkd

• If snapservice is not used:

1. Identify the Docker bridge interface using the below command:

# ifconfig

root@source_vr [ /var/log/vmware/hms ]# ifconfig

br-9a3cb6593d00: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.#.#  netmask 255.255.0.0  broadcast 172.17.255.255
        ether b6:de:a3:##:##:##  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

2. Bring down the unused bridge interface:

# ip link set br-9a3cb6593d00 down