Guidance on performing a Top-of-Rack (TOR) switch upgrade maintaining virtual machine connectivity and ensure cluster availability.

Products

• VMware NSX

Issue/Introduction

 This article provides a comprehensive pre-upgrade checklist and a validated sequential restart method for redundant switch configurations.

Environment

• VMware vsphere
• VMware NSX

Resolution

Perform the following pre-upgrade checks and sequential upgrade steps to ensure a smooth transition without service interruption.

Pre-Upgrade Checklist

1. Confirm that every ESXi host has active uplinks to both TOR switches in the vSphere Client under Host > Configure > Virtual Switches.
2. Verify that the vSAN health status is Green with no network or physical network alarms by navigating to vSAN Cluster > Monitor > Skyline Health.
3. Validate that the MTU settings (e.g., Jumbo Frames 9000) on the switch firmware match your existing vSAN VMkernel adapter configuration.
4. Perform a manual failover test on a host in Maintenance Mode (Ensure Accessibility):
   • Disable one uplink: esxcli network nic down -n vmnic####.
   • Confirm traffic continues through the remaining active uplink.
   • Re-enable the NIC: esxcli network nic up -n vmnic####.
   • Repeat the steps for the other nic and confirm traffic continues through the remaining active uplink.

Sequential Upgrade Procedure

1. Upgrade and restart Switch A first.
2. Wait for the switch to return to a fully operational state and confirm that all ESXi hosts show the associated links as Up.
3. Verify that vSAN health remains stable with no partitions.
4. After you confirm Switch A is stable, proceed with the upgrade for Switch B.

Additional Information

You can also refer to the kb: https://knowledge.broadcom.com/external/article/393205  for more information on "Network changes on a vSAN cluster with out impacting vSAN storage"