Unable to update Transport Node Profile on ESXi Transport Node if VMs are running on that host

Products

• VMware NSX

Issue/Introduction

Symptoms:

• Updates to ESXi Host Transport Node and Transport Node Profile attached to a host with powered on VMs fail with the error: "The host has powered on VMs which must be moved or powered off before transport node create/update/delete can continue"
• Updates to an ESXi host Transport Node (TN) will fail if it has VMK migration specified and there are any powered-on VMs on that ESXi host.
• Updates to a Transport Node Profile (TNP) attached to such TNs will fail regardless of the VMK migration setting on the TNP.

Environment

VMware NSX Transformers 2.x

Cause

This issue is caused because powered-on VMs cause the migration validation to fail, preventing updates to the TN or TNP.

Resolution

Currently, there is no resolution.

Workaround:
There are multiple scenarios to workaround this issue:

Scenario 1:

You have a TNP attached to the compute cluster and you are able to put either all the hosts in that compute cluster or one host at a time into maintenance mode.

Follow these steps:

1. In vCenter server: Put all the hosts or one host in the cluster into maintenance mode.
2. In NSX Manager: Edit the TNP configuration and apply to the compute cluster as follows:
   1. From System > Fabric > Profiles > Transport Node Profiles, select the TNP and click Edit. Make your changes and click Save.
      Note: If you are updating one host at a time, updated TNP configurations will succeed on the host which is in maintenance mode but will fail on other hosts that are not in maintenance mode. This is an expected behaviour.
   2. From System > Fabric > Host Transport Node, select the cluster attached to this TNP and click Configure NSX. The TNP should be selected in the dialog box that appears. Click Save to apply the TNP edits to the cluster.
3. In vCenter server: Take the hosts, or the one host you edited, in the cluster out of maintenance mode.
4. Only required if you are updating one host at a time: Repeat steps 1 through 3 for each additional host in the cluster that has not been put into maintenance mode. When you reapply the TNP configuration on the cluster, you will still see errors regarding powered on VMs even for hosts already edited that you brought out of maintenance mode. You can ignore these errors.  If you do not want to see these errors, detach the TNP from the cluster.   ​​​​​​

Scenario 2:

You have not attached a TNP to the compute cluster and you are able to put the ESXi host TN in maintenance mode.

Follow the steps:

1. In vCenter Server: Go to the ESXi host with powered-on VMs and put it in maintenance mode.  
2. In NSX Manager: Update that ESXi host transport node and apply the changes as follows:
   1. From System > Fabric > Host Transport Node, select the transport node and click Configure NSX.
   2. On the Install NSX wizard, click Next to the Configure NSX screen. Make the required edits and click Finish.
3. In vCenter Server: Take the ESXi host out of maintenance mode.

Scenario 3:

You are not able to put any hosts in maintenance mode.

Follow these steps:

1. If you are using Transport Node Profile, detach the Transport Node Profile from compute cluster that contains Transport Node that you want to update. Detaching the Transport Node Profile from compute cluster will not change configurations already applied to any host in the cluster.
2. For each TN that you want to update, remove the VMK install mapping. This allows bypassing the migration validation that requires the host to have no powered-on VMs.  VMKs that   have already migrated to N-VDS are not affected by this change. After the update, do not add back the VMK install-mapping to the TN, to prevent this issue from recurring.
   1. Navigate to System > Fabric > Nodes >  Host Transport Nodes.  
   2. Select the TN you want to update and click "Configure NSX".
   3. In the NSX Installation wizard, click "Next" to go to the Configure NSX step.
   4. Scroll down to the section that shows "Network Mappings for Install" and remove the existing mappings.
   5. Click "Finish". 
3. Edit the TN configuration as required.

Note: If the NSX-T hostswitch (N-VDS) was configured with "Network Mappings for Install" (that are removed now as part of this workaround), do not enable the "PNIC only Migration" flag. To migrate PNICS that are not currently in use, add these PNICs as uplinks to the PNICs list in the Configure NSX screen of the NSX Installation wizard. If the PNICs are currently in use by VSS/DVS, first free them up with