DVS Status Shows as "Down" When we select Adding/Managing Hosts on Distributed Virtual Switch (DVS) after ESXi Host reboot.

Products

• VMware NSX
• VMware vSphere ESXi

Issue/Introduction

• You may observe the Distributed Virtual Switch (DVS) status showing as "Down" when attempting to add or manage an ESXi host on the DVS. This issue can prevent proper network configuration and host connectivity through the DVS.
• Host was rebooted recently after NSX-V to NSX-T Migration.
• Unable to vMotion the VM's.
  
  

Environment

VMWare NSX
VMware vSphere ESXi 7.x
VMware vCenter Server 7.0.x
VMware vSphere ESXi 8.x
VMware vCenter Server 8.0.x

Cause

This issue is observed due to stale NSX-V entries in following scenario :

• Environment had NSX-V running with VXLAN feature enabled on VDS 7.0 and later migrated the system to NSX-T using NSX V2T migrator
• This might leave some stale NSX-V properties in the VDS in some or all hosts and host upgrade fails.

Resolution

This is a known issue, please follow the steps in workaround section to resolve this issue.

Workaround
Follow below steps to workaround this issue :

1. Download the CleanNSXV.py Python script attached to this KB
2. Upload the script to path /tmp/ on vCenter Server Appliance Using WinSCP. Please refer KB Error when uploading files to vCenter Server Appliance using WinSCP if WinSCP fails to connect to VCSA.
3. Login to VCSA with root credentials using any SSH Client (Eg. Putty)
4. Run the CleanNSXV.py script as mentioned below :

PYTHONPATH=$VMWARE_PYTHON_PATH python /tmp/CleanNSXV.py --user '<vc_admin_user>' --password '<passwd>'

Note: The <vc_admin_user> parameter is a vCenter Server user with super administrative privileges and <passwd> parameter is the user password.

For example:
PYTHONPATH=$VMWARE_PYTHON_PATH python /tmp/CleanNSXV.py --user '[email protected]' --password '#########'

Sample Results:

Sample output of the script is below in which 2 hosts are listed:
Namespace(password='##########', user='[email protected]')
Connecting to vCenter service
Connected to vCenter service
Cleaning VDS 50 06 6f 76 ## ## ## ##-## ## ## ## c8 e4 00 82 ...
Begin to clean up NSXV properties on host[1x.xx.xx.xx]...
NSXV properties get cleaned up on host[1x.xx.xx.xx].
Begin to clean up NSXV properties on host[1x.xx.xx.xx]...
NSXV properties get cleaned up on host[1x.xx.xx.xx].
Cleaned VDS 50 06 6f 76 ## ## ## ##-## ## ## ## c8 e4 00 82


If no more VDS has the stale NSX-V properties, the output is as below:
Namespace(password='#########', user='[email protected]')
No VDS needs clean up

5. Verify if NSX-V properties "com.vmware.netoverlay.layer0" and "com.vmware.net.vxlan.udpport" are removed from the ESXi hosts: (Note: Select some random hosts to do the verification from the list of hosts showed in the script result from Step 4, no need to perform this test on all the hosts)
   • Connect to an ESXi host by using an SSH client.
   • Run the command 'net-dvs -l | grep "com.vmware.netoverlay.layer0\|com.vmware.net.vxlan.udpport"'.
   • If you see no output, then the stale properties are removed.

Additional Information

If Only one Host is affected the workaround provided in the KB DVS Status Shows as "Down" When we select Adding/Managing Hosts on Distributed Virtual Switch (DVS).