vSAN network migration procedure across isolated physical switch fabrics
Article ID: 441523
Updated On:
Products
Issue/Introduction
Administrators need to migrate vSAN network traffic from legacy physical switches to new physical switches. The legacy and new physical switch fabrics are isolated and do not communicate with each other at Layer 2. Attempting a standard rolling migration causes the vSAN nodes to lose unicast communication across the isolated fabrics, resulting in an immediate cluster partition and All Paths Down (APD) conditions for vSAN objects.
Environment
VMware vSAN 8.0 U3
Cause
The physical network fabrics lack an Inter-Switch Link (ISL), creating isolated broadcast domains. A rolling migration breaks vSAN unicast communication because nodes on the legacy switch cannot route traffic to nodes on the new switch.
Resolution
Gracefully shut down all virtual machines running on the target vSAN cluster. If the vCenter Server VM resides on this cluster, migrate it to a different cluster prior to beginning, or prepare to execute the network cutover locally via the ESXi Host Client.
Place all ESXi hosts in the cluster into Maintenance Mode simultaneously using the No Data Migration policy.
Once all hosts are in Maintenance Mode, reconfigure the vSAN port group (via Standard Switch or Distributed Switch) on every host to utilize the new physical uplinks. Explicitly set the legacy uplinks to Unused.
Open an SSH session to each host and verify vSAN network flow across the new isolated fabric using:
vmkping -I <vSAN_vmk_interface> -s 8972 -d <target_host_vSAN_IP>. (Omit the-s 8972flag if Jumbo Frames are not configured).After confirming 0% packet loss between all vSAN nodes on the new fabric, exit Maintenance Mode on all hosts simultaneously. The vSAN cluster will automatically reform.
Navigate to vSAN Skyline Health, execute a manual re-test, and verify that Network Health and Data Object Health report a fully healthy state.
Power on the virtual machines.
Additional Information
Feedback
