Uninstalling NSX-T from a host transport node fails: Unable to clear DVS propertycom.vmware.nsx.vdl2.enabled

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

• Removing NSX gracefully from the UI fails.
• Removing NSX from the ESXi host directly via the NSXCLI with the 'nsxcli -c del nsx' command fails.
• On the ESXi host the following vDS property is observed to be RUNTIME instead of expected CONFIG

#net-dvs | grep "com.vmware.common.opaqueDvs.status.component.vswitch"
          com.vmware.common.opaqueDvs.status.component.vswitch = up ,     propType = RUNTIME

• In the NSX-T manager log '/var/log/syslog' we see the following entries during failed removal from the NSx-T UI:

Host configuration: host switch remove operation failed: [Failed to hot-swap cvds [\'50 0d a0 72 e3 9d 14 44-c3 e2 bc a7 a5 e0 5d 98\']ApplyNsxDvsConfig on (50 0d a0 72 e3 9d 14 44-c3 e2 bc a7 a5 e0 5d 98) failed: {\'fault\': \'PlatformConfigFault\', \'msg\': \'An error occurred during host configuration.\', \'faultMessage\': [\'Operation failed, diagnostics report: Unable to clear DVS propertycom.vmware.nsx.vdl2.enabled: Status(bad0004)= Busy\']}];  LogicalSwitch full-sync: LogicalSwitch full-sync realization query skipped.

• In the ESXi log /var/run/log/nsxcli.log after we run the 'del nsx' command we see the following entry:

% Exception when deleting nsx from host: ' error code: 4 stdout: delete_nsx_instance_from_host.sh: INFO: NSX reset script called with argument fabric_node on nsx-esx delete_nsx_instance_from_host.sh: INFO: Run transport_node reset on ESX node % Failed to remove all host switches or logical switches delete_nsx_instance_from_host.sh: ERROR: Failed to reset nsxa app of nsx-opsagent. Please check ospagent logs for more details. , stderr: Tue May 02 2023 UTC 15:00:54.002 ERROR: Failed to reset nsxa app of nsx-opsagent. Please check ospagent logs for more details.'

Environment

VMware NSX-T Data Center 4.x
VMware NSX-T Data Center

Cause

This issue occurs due to a change in API behaviour when updating the status of vDS properties.

Resolution

This is a known issue impacting VMware NSX.

Workaround:
Execute the following commands from the ESXi shell:
 

1. Log into the shell of the impacted ESXi host
2. Check the vDS name

      #net-dvs | grep "com.vmware.common.alias"

3. Correct the problem vDS property
    #net-dvs -s com.vmware.common.opaqueDvs.status.component.vswitch=up -p hostPropList <vDS Name>

               <vDS Name> from step 2

4. Now continue with uninstall of the host if that was the desired objective.