Cumulus 3.3 or later causes Network Issues due to the Reserved VLAN Range on the Management Switch in VMware Cloud Foundation 2.3.x

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• The management switch for your VMware Cloud Foundation implementation was recently updated to version 3.3 or later of the Cumulus operating system.
• The VMware Cloud Foundation version is 2.3.x or higher.
• There are management, datacenter, VSAN, VXLAN, or VMotion VLANs definded in the range of 3000-3999.
• The management switch is unreachable via IP address from virtual machines or hosts.

Environment

VMware Cloud Foundation 2.3.x

Cause

By default, Cumulus 3.3 and later reserves 1000 VLANs in the range of 3000-3999 to accommodate internal data plane operations. If VLANs have been defined in this range for any of the workload domains, networking issues will occur when attempting to communicate with the management switch from any of the VLANs that overlap from this range.

Resolution

To resolve this issue, complete the following steps:

1. Connect to the management switch via SSH or using a console connection.
2. Edit the following file using a text editor: /etc/cumulus/switchd.conf
3. Remove the hash sign (i.e., the '#' character) from the beginning of the line that starts with resv_vlan_range. The line will be similar to the following:
   
   #resv_vlan_range = 3000-3999
    
4. Adjust the range to exclude the VLANs that are configured in the environment. The IDs can be anywhere between 2 and 4094 and must contain a minimum of 300 VLANs. The following would be an example of a line adjusted to exclude anything below 3700:
   
   resv_vlan_range = 3700-3999
    
5. Restart the switchd.service on the switch with the following command:
   
   sudo systemctl restart switchd.service

Additional Information

The changes to the switchd.conf can also be completed using the following command:

sed -i "s/#resv_vlan_range = 3000-3999/resv_vlan_range = 3700-3999/g" /etc/cumulus/switchd.conf;reboot

Please note, this command will restart the switch.

Additional information regarding the reserved VLANs can be found at the following location:
https://docs.cumulusnetworks.com/display/DOCS/VLAN-aware+Bridge+Mode+for+Large-scale+Layer+2+Environments