NSX-T migration from NSX for vSphere to NSX-T fails with vdl2 down error

Products

• VMware NSX

Issue/Introduction

• Performing a v2t migration that will migrate ESX hosts.
• When having one or more VXLAN enabled cluster's in NSX for vSphere and one or more Firewall enabled clusters with no VXLAN in NSX for vSphere.
• The VDS used by one cluster for VXLAN is also used by a cluster that has only Firewall enabled.
• During migration, the workloads VMs on the clusters that have only Firewall enabled fail to migrate with error message similar to below:

Pre-migrate stage failed during host migration [Reason: [Vmotion] Can not proceed with migration: Host failed to enter maintenance mode due to the following DRS faults ['DRS failed to migrate VM test-vm due to faults ["Currently connected network interface \'Network adapter 1\' uses network \'DVSwitch[50 0b c5 af b1 04 97 4a-97 72 a4 3d 08 45 b4 fc] NSX port group [dvportgroup-700009](vdl2 down)\', which is not accessible."]

• Some of the host in the cluster did get migrated from NSX for vSphere to NSX-T, but at least one host has failed to migrate.
• Checking the ESXi host that was migrated, using command 'esxcfg-vmknic -l' & see a link local IP address assigned to the TEP interface e.g 169.254.x.x

Environment

VMware NSX-T Data Center

Cause

• In such a setup, the migration coordinator will migrate a cluster that has only Firewall enabled as if it had VXLAN enabled and create TEPs with DHCP in each host in the cluster.
• As there is no DHCP configured to assign IP addresses to these vmkernels, they end up getting the link local addresses.
• As these link local addresses are unable to communicate with the NSX-T managers, this will cause the vdl2 service to go down.
• Once the vdl2 service is down, vMotions will not be allowed. Hence the reason the above error is seen. The migration coordinator cannot proceed to migrate the next host, as it cannot vMotion workloads of the host to other host(s) that were already migrated, due to vdl2 being down.

Resolution

This is a known issue in NSX-T datacenter.

Workaround:
Edit the transport host-switch uplink profile to use an IP pool, this will allow to get a correct IP address and allow vdl2 to start which can manually migrate the VMs and continue the migration.
Or if there is not enough IPs to assign to the TEPs, or if the hosts in a cluster that has only Firewall enabled do not need to support overlay, edit each transport node in the cluster to remove the overlay transport zone id, wait till the transport node state becomes success and then retry the migration.