NVDS to CVDS migration fails because a port can not be removed on ESXi

Products

• VMware NSX Networking

Issue/Introduction

• NVDS to CVDS migration fails on a ESXi.
• You see NVDS deletion failed because a port being busy can not be removed in the ESXi logs.
  nsxaVim.log

  <TimeSamp> nsxaVim: [<NNNNNNN>]: ERROR Dvs [<DVS ID>] cannot be deleted; Failed to delete dvs [<DVS ID>]: {'fault': 'PlatformConfigFault', 'msg': 'An error occurred during host configuration.', 'faultMessage': ['Operation failed, diagnostics report: Unable to Remove Port; Status(bad0004)= Busy']}

  vmkernel.log

  <TimeSamp> cpu<NN>:<NNNNNNN> opID=<opID>)NetDVS: 772: Failed to remove busy port <DVS ID> <Port UUID>, type 1, port->refcnt 0[reserved] port->portID <Hex Port ID>

Environment

VMware NSX-T Datacenter 3.x

Cause

Ports being busy block NVDS to be deleted.

It is known that some operations like force-deleting a port could result in such situation.

Resolution

First, reboot the ESXi to release the ports.

To resume or complete migration, follow the steps.

• If the host is the last one in the cluster
  

  1. Create a VDS-TNP from NVDS-TNP:

     POST https://{{nsx_ip}}/api/v1/transport-node-profiles/{{NVDS-TNP-ID}}?action=migrate_to_vds&ref_tn_id={{tn-id}}

     tn-id is the TN uuid for which the migration has failed.
     NVDS-TNP-ID is the pre-migration TNP ID.

  2. The API generates a new TNP with name MigratedTNP-{{Old-TNP-Name}}.

     Verify that the new TNP configuration matches with the configuration of TNs under the cluster.

  3. Apply the new TNP on the cluster.
  4. Clean-up the precheck with following API:

     PUT https://{{nsx_ip}}/api/v1/nvds-urt?action=cleanup

     Now the migration of the cluster has been completed.
  5. If any other cluster is to be migrated, start precheck again.
• If any other host in the cluster is to be migrated
  1. Clean-up the precheck with following API:

     PUT https://{{nsx_ip}}/api/v1/nvds-urt?action=cleanup

  2. Re-run precheck again and resume the migration.