How to change the default Docker 172.17.0.0 range in VMware Cloud Foundation 9.0 Ops Fleet Manager to avoid network conflicts with customer infrastructure

Products

VCF Operations VMware Cloud Foundation

Issue/Introduction

VCF Ops Fleet Manager in VCF 9.0 comes with Docker pre-installed, which uses a default bridge network CIDR range of 172.17.0.0/16.

This default range may conflict with customer network infrastructure, including:

VCF Ops Fleet Manager services cannot communicate properly
Docker containers cannot reach external services
Network routing conflicts in the environment
Connection timeouts or failures from containerized services
vCenter or cloud proxy communication issues when their networks overlap with 172.17.0.0/16

Environment

VCF Operations 9.x
VMware Cloud Foundation 9.x

Cause

Network conflicts can prevent VCF Operations Fleet Manager from functioning correctly, causing communication failures and service disruptions.

Resolution

Before proceeding, ensure you have:

Root or sudo access to the VCF Ops Fleet Manager appliance
SSH access enabled on the Ops Fleet Manager
Maintenance window scheduled (requires Docker service restart)
Network CIDR range selected that does NOT conflict with:
- vCenter Server networks
- ESXi management networks
- Cloud proxy networks
- Any other infrastructure networks
Backup of the VCF Ops Fleet Manager (recommended)

Resolution

Phase 1: Pre-Change Assessment

Step 1: Identify Current Network Usage

# SSH to VCF Ops Fleet Manager appliance
ssh root@<ops-fleet-manager-ip>

# Check current Docker network configuration
docker network ls
docker network inspect bridge

# Verify the current Docker bridge IP
ip addr show docker0

# Check current routing table
ip route show

# List running containers (note these for post-change verification)
docker ps -a

Expected Output: You should see 172.17.0.0/16 as the current bridge network range.

Step 2: Choose Non-Conflicting CIDR Range

Select a CIDR range that doesn't conflict with your infrastructure. Common alternatives:

Range	CIDR	Use Case
10.x.x.x	10.200.0.0/16	Large enterprises, less likely to conflict
192.168.x.x	192.168.100.0/16	Small to medium environments
172.x.x.x	172.20.0.0/16	Alternative 172 range (avoid 172.16-172.31 if used elsewhere)

Important: Verify the chosen range is not used in your environment:

# Check for routing conflicts
ip route | grep <your-chosen-range>

# Example: Check if 10.200.0.0/16 is in use
ip route | grep 10.200

Phase 2: Backup Current Configuration

Step 3: Create Backup

# Create backup directory
sudo mkdir -p /etc/docker/backup

# Backup existing daemon.json if it exists
sudo cp /etc/docker/daemon.json /etc/docker/backup/daemon.json.backup 2>/dev/null || echo "No existing daemon.json found"

# Backup Docker's internal state
sudo cp -r /var/lib/docker/network /etc/docker/backup/network.backup 2>/dev/null || echo "No network state to backup"

Phase 3: Change Docker Bridge Network Configuration

Step 4: Stop Docker Service

# Stop Docker service
sudo systemctl stop docker

# Verify Docker is stopped
sudo systemctl status docker

Step 5: Configure New CIDR Range

Option A: Create New Configuration (if /etc/docker/daemon.json doesn't exist)

# Create daemon.json with new CIDR range
# Example using 192.168.100.0/16 range
sudo tee /etc/docker/daemon.json > /dev/null <<EOF
{
  "bip": "192.168.100.1/24",
  "default-address-pools": [
    {
      "base": "192.168.100.0/16",
      "size": 24
    }
  ]
}
EOF

Option B: Modify Existing Configuration

# First, check existing configuration
sudo cat /etc/docker/daemon.json

# Backup the current file
sudo cp /etc/docker/daemon.json /etc/docker/daemon.json.pre-change

# Edit the configuration
sudo vi /etc/docker/daemon.json

Add or modify these parameters (preserve existing settings):

{
  "bip": "192.168.100.1/24",
  "default-address-pools": [
    {
      "base": "192.168.100.0/16", 
      "size": 24
    }
  ]
}

Configuration Parameters Explained:

bip (Bridge IP): Sets the IP address and netmask for the docker0 bridge interface
default-address-pools: Defines address pools for user-defined bridge networks
- base: The subnet range for the pool
- size: The CIDR netmask size for individual networks (24 = /24 subnet)

Step 6: Validate Configuration

# Validate JSON syntax
python3 -c "import json; json.load(open('/etc/docker/daemon.json'))" && echo "JSON is valid" || echo "JSON syntax error!"

# Display the configuration
cat /etc/docker/daemon.json

Phase 4: Apply Changes and Verify

Step 7: Start Docker Service

# Start Docker service
sudo systemctl start docker

# Check service status
sudo systemctl status docker

# If Docker fails to start, check logs:
sudo journalctl -u docker.service --no-pager -l | tail -50

Step 8: Verify New Configuration

# Verify new bridge network configuration
docker network inspect bridge | grep -A 5 "IPAM"

# Check docker0 interface has new IP
ip addr show docker0

# Expected output should show your new CIDR range (e.g., 10.200.0.1/24)

# Verify routing table
ip route | grep docker0

# Check Docker daemon is healthy
docker info | grep -A 10 "Server Version"

Expected Results:

Docker0 bridge should have the new IP address (e.g., 10.200.0.1/24)
IPAM Config should show the new subnet range
Docker daemon should be running without errors

Verification Checklist

After completing the procedure, verify:
Docker service is running: systemctl status docker
Docker0 bridge has new IP: ip addr show docker0
New CIDR range is configured: docker network inspect bridge
No routing conflicts: ip route
Containers are running: docker ps

Additional Information

Troubleshooting

Issue 1: Docker Service Fails to Start

Symptoms:

Failed to start Docker Application Container Engine.

Resolution:

# Check Docker logs for specific error
sudo journalctl -u docker.service --no-pager -l | tail -100

# Common causes:
# 1. Invalid JSON syntax in daemon.json
python3 -c "import json; json.load(open('/etc/docker/daemon.json'))"

# 2. Network range conflict
ip route | grep <your-new-range>

# 3. Permission issues
ls -l /etc/docker/daemon.json
sudo chmod 644 /etc/docker/daemon.json

# 4. Restore backup and retry
sudo systemctl stop docker
sudo cp /root/docker-backup-*/daemon.json.backup /etc/docker/daemon.json
sudo systemctl start docker

Issue 4: Network Still Shows Old CIDR Range

Symptoms:

docker network inspect bridge shows old 172.17.0.0/16 range

Resolution:

# This may happen if networks are cached
# Stop Docker completely
sudo systemctl stop docker

# Remove old network state (CAUTION: This will reset all networks)
sudo rm -rf /var/lib/docker/network/*

# Start Docker
sudo systemctl start docker

# Verify new configuration
docker network inspect bridge