VMware NSX 4.1.0 docker network subnet can conflict with workload subnet

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

• You are using VMware NSX 4.1.0.
• You have medium or large NSX Managers deployed.
• The internal docker0 interface subnet 172.17.0.1/16 on the NSX Manager conflicts with your existing subnets.
• You can confirm the docker0 interface is present by running ifconfig from root of the NSX manager:

root@nsx-mngr-01:~# ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
    inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
    ether 02:42:e5:17:ff:ae txqueuelen 0 (Ethernet)
    RX packets 0 bytes 0 (0.0 B)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 0 bytes 0 (0.0 B)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

NOTE: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware NSX-T

Cause

Docker is installed in VMware NSX 4.1.0, when deployed as Medium or Large size appliance only. The default network is 172.17.0.1/16.

Resolution

The automatic install of the docker0 interface will be removed from a future version VMware NSX.

Workaround:
There are 2 options below to workaround the conflict.
Make sure you have a up to date VMware NSX backup in place.

1. Do not create docker bridge 'docker0': 

- login as root to an NSX Manager.
- Edit the following file /etc/docker/daemon.json or create it if it does not exist. Add below json :

root@nsx-mngr-01:/etc/docker# vim daemon.json

{
"bridge": "none"
}

- Restart the docker service:
root@nsx-mngr-01:/etc/docker# systemctl restart docker
 

2. Change docker0 subnet:

- login as root to an NSX Manager.
- Edit the following file /etc/docker/daemon.json or create it if it does not exist. Add below json :
root@nsx-mngr-01:/etc/docker# vim daemon.json

{
"bip": "172.26.0.1/16" >>> Enter the subnet you would like to use here
} :

- Restart the docker service:
root@nsx-mngr-01:/etc/docker# systemctl restart docker

As the docker service is not in use in this version, on the NSX Manager, you can stop containers using below command:

root@nsx-mngr-01:/etc/docker# systemctl stop nsx-metrics-agents.service nsx-datacollector-agents.service nsx-kafka.service

These services will get started after reboot again, to disable them use below command:

root@nsx-mngr-01:/etc/docker# systemctl disable nsx-metrics-agents.service nsx-datacollector-agents.service nsx-kafka.service