VMware Edge Docker network subnet can conflict with workload subnet
search cancel

VMware Edge Docker network subnet can conflict with workload subnet

book

Article ID: 415157

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • When running VMware NSX Edge Node 4.x, you may experience routing conflicts if your existing workload subnets overlap with the internal Docker interface.

    By default, the internal docker0 interface on the Edge node uses the subnet 169.17.0.0/16. If your environment utilizes this same subnet for workloads, it will cause a conflict.

    To confirm the configuration of the docker0 interface, log in as root to the Edge node and run the ifconfig command:

    root@edge01:~# ifconfig
    docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
            inet 169.17.0.1  netmask 255.255.0.0  broadcast 169.17.255.255
            inet6 fe80::42:9bff:fed6:8493  prefixlen 64  scopeid 0x20<link>
            ether 02:42:9b:d6:84:93  txqueuelen 0  (Ethernet)
            RX packets 3  bytes 180 (180.0 B)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 2  bytes 196 (196.0 B)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Note: The preceding log excerpts are only examples. Date, time, MAC addresses, and specific environmental variables may vary depending on your setup

Environment

VMware NSX 

Cause

Starting in the NSX 4.x release, Docker is installed on the VMware NSX EDGE, it automatically provisions a default docker0 interface utilizing the "169.17.0.0/16" subnet space.

Resolution

  • This issue is resolved in VMware NSX 9.1.1. Upgrading to this version or later will address the conflict.

  • If you are unable to upgrade immediately, you can resolve the conflict by changing the docker0 subnet to a non-overlapping range.

    1. Log in as root to the affected Edge node.

    2. Edit the Docker daemon configuration file located at /etc/docker/daemon.json. (If the file does not exist, create it).

    3. Add the following JSON configuration, replacing the IP address with a non-conflicting subnet appropriate for your environment:

root@Edge-01:/etc/docker# vim daemon.json
{
"bip": "169.17.0.1/16" >>> Enter the subnet you would like to use here
} :
- Restart the Docker service to apply the changes:
root@Edge-01:/etc/docker# systemctl restart docker

Note: If you place the Edge node into Maintenance Mode prior to making these modifications, you do not need to manually restart the Docker service.

  • Alternatively, you can add static routes via the NSX Management Plane (MP) API to force traffic for the overlapping IPs through the management interface (eth0).

    Use the following API call as a reference. Ensure you replace the placeholder values (<admin-user>, <password>, <nsx-mgr-ip>, <transport-node-id>, and <overlapping-ip>) with your specific environmental details:

    curl -k -X POST -u '<admin-user>':'<password>' -H 'Content-Type: application/json' https://<nsx-mgr-ip>/api/v1/transport-nodes/<transport-node-id>/node/network/routes --data '
    {
      "route_type": "static",
      "destination": "<overlapping-ip>",
      "netmask": "255.255.255.255",
      "interface_id": "eth0"
    }'

    API Response:
    {
      "_schema": "NodeRouteProperties",
      "_self": {
        "href": "/transport-nodes/<transport-node-id>/node/network/routes",
        "rel": "self"
      },
      "destination": "<overlapping-ip>",
      "interface_id": "eth0",
      "ipv6": false,
      "netmask": "255.255.255.255",
      "proto": "boot",
      "route_id": "static-<overlapping-ip>-eth0-0",
      "route_type": "static"
    }

 

Additional Information

Powered by Wolken Software