Using a management interface other than vmk0 causes failure of migration from N-VDS to VDS

Products

• VMware NSX

Issue/Introduction

Symptoms:

• You're running NSX 3.x
• You're attempting to preform a migration from N-VDS to VDS
• One or more of the ESXi hosts to be migrated are configured to have a management vmkernel other than vmk0
• When you run the command esxcfg-vmknic -l you see vmk1 being used as management interface.

vmk1 73eff72a-xxxx-xxxx-xxxx-116740613ba4 IPv4 <Management-IPv4> <net-mask> <Broadcast> <MAC-addr-01> 1500 65535 true STATIC defaultTcpipStack
vmk1 73eff72a-xxxx-xxxx-xxxx-116740613ba4 IPv6 <Management-IPv6> <net-mask> <MAC-addr-01> 1500 65535 true STATIC, PREFERRED defaultTcpipStack
vmk0 9a7e4e1b-xxxx-xxxx-xxxx-8a8397beff78 IPv4 <Different-IPv4> <net-mask> <Broadcast> <MAC-addr-02> 1500 65535 true STATIC defaultTcpipStack
vmk0 9a7e4e1b-xxxx-xxxx-xxxx-8a8397beff78 IPv6 <Different-IPv4> <net-mask> <MAC-addr-02> 1500 65535 true STATIC, PREFERRED defaultTcpipStack

• You may observe the following log entries:

/var/log/nsx-syslog.log
yyyy-mm-ddThh:mm:ss nsx-opsagent[2451059]: NSX 2451059 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="nsxa" tid="2451320" level="ERROR" errorCode="MPA42018"] [MigrateToVSS] MigrateOutofLS(<uuid>) failed: [socket read failed]

/var/run/log/nsxaVim.log
yyyy-mm-ddThh:mm:ss nsxaVim: [4437820]: ERROR Server: got Exception: Traceback (most recent call last): File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/nsxaVim.py", line 3946, in Server result = ProcessCommand(data.decode('utf-8')) File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/nsxaVim.py", line 3704, in ProcessCommand return hostConfig.MigrateToVSS(rootFolder, networkSystem, dvsManager, logger, cmd[13:-1]) File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/hostConfig.py", line 775, in MigrateToVSS status = MigrateToVSSInt(rootFolder, networkSystem, dvsManager, dvsUuid, File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/hostConfig.py", line 1166, in MigrateToVSSInt proxySwitchInfo = MigrateToProxySwitch(networkSystem, dvsManager, vnics, portgroups, File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/hostConfig.py", line 2572, in MigrateToProxySwitch error, vmksMigrate2AnyPnic = AssignPnic2UplinkForManagementVmksMigrating2AnyPnic( File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/hostConfig.py", line 330
...
yyyy-mm-ddThh:mm:ss nsxaVim: 0, in AssignPnic2UplinkForManagementVmksMigrating2AnyPnic     port = GetFreeUplinkPortByUplinkKeyAndMarkAsInUse(dvsPortsInfo, minUplink.name)   File "/usr/lib64/vmware/nsx-opsagent/pyvim/nsxa/hostConfig.py", line 2267, in GetFreeUplinkPortByUplinkKeyAndMarkAsInUse     dvsPortsInfo.uplinksFree.remove(port.name) KeyError: 'uplink-1'^@

Environment

VMware NSX-T Data Center 3.x
VMware NSX-T Data Center

Cause

For N-VDS to VDS migration, it's recommended to have the vmkernel with the smallest index enabled with management service.

Resolution

If you still haven't started the migration, please reconfigure the host to have vmk0 as management interface.

Workaround:
If you have already started the migration, the workaround would be as below:

1. Apply the old Transport Node Profile, which was present before starting NVDS to CVDS migration, on the cluster where the issue exists.
2. Wait for transport node state to be successful.
3. Enable management service on vmk0 from vCenter UI, if it is disabled.
4. Move the existing management IP configurations to vmk0.
5. Make sure the transport node state shows as success
6. Clean up old URT precheck using the following API: POST /api/v1/infra/nvds-urt?action=cleanup
7. Start a new migration procedure per the following document: Migrate Host Switch to vSphere Distributed Switch