How to change kind-cluster IP network range in bootstrap VM

Products

• VMware

Issue/Introduction

This KB provides how to change the docker default network after failing the bootstrap of TKG.

Symptoms:
Docker's default network range is 172.17.0.0/16.  

If the existing external network includes 172.17.0.0/16, the bootstrap VM can't forward the packet to external network 172.17.0.0/16.  As a result, TKG bootstrap is failed.

Environment

VMware Tanzu Kubernetes Grid 1.x

Cause

Depending on the customer environment, Docker's default network range 172.17.0.0/16 conflicts with the existing external network because 172.17.0.0/16 is vast.

Resolution

Change the default docker network configuration.

# SSH to bootstrap VM
ssh root@${BOOTSTRAP_VM}

# Stop all running docker containers
docker ps
docker stop $CONTAINER_ID

# Check current IP address list
ip a

# Delete target bridge which was generated by kind
ip link set dev br-xxxx down
ip link del dev br-xxxx

# Delete the target bridge which was generated by kind
docker network ls
docker network rm kind

# Set non conflict network range
cat > /etc/docker/daemon.json <<EOF
{
    "default-address-pools":[  
        {  
            "base":"172.40.0.0/24",
            "size":27
        }
    ]
}
EOF

# Restart docker
systemctl restart docker
systemctl status docker

# Check IP address on docker0
ip a show dev docker0
#> inet 172.40.0.1/27 brd 172.40.0.31 scope global docker0

# Check: routing table
ip route

# Test (install "kind" in advance)
kind create cluster -n test

# Check new created bridge IP address
ip a show | grep br- | grep -B1 inet
#>13: br-d1811fcbec05: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
#>    inet 172.40.0.33/27 brd 172.40.0.63 scope global br-d1811fcbec05

Additional Information

Impact/Risks:
TKG bootstrap process is failed.