NVDS to VDS migration URT precheck fails due to missing vmk logical port on NSX MP
search cancel

NVDS to VDS migration URT precheck fails due to missing vmk logical port on NSX MP

book

Article ID: 312973

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • URT precheck for NVDS to VDS migration fails with following error messages
"precheck_id" : "<precheck-ID>",
"precheck_status" : "FAILED"
Get VDSUpgradeSummary: {"precheck_id": "<precheck-ID>", "precheck_status": "FAILED"}

 

  • Entries similar to the below are observed in the NSX Manager log var/log/proton/nsxapi.log for the problematic transport node and vmk interface
var/log/proton/nsxapi.log

2025-03-10T06:30:31.683Z  INFO NvdsUpgradeTaskExecutor1 NvdsUpgradeReadinessCheckServiceImpl 4858 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] NVDS-UPGRADE TN <Transport-Node-UUID> vmk id {
  value: "vmk1"
}
link_status: INTERFACE_STATUS_UP
admin_status: INTERFACE_STATUS_UP
mtu: 9000
interface_type: INTERFACE_TYPE_VIRTUAL
connected_switch {
}
interface_entry {
  broadcast_address {
    value: "<Broadcast IP>"
  }
  ip_address {
    value: "<IP>"
  }
  ip_config: IP_CONFIG_STATIC
  netmask {
    value: "<Subnet Mask>"
  }
  mac_address {
    value: "<MAC address>"
  }
}
host_managed: true
lport_attachment_id {
  value: "<Logical-Port-Attachment-ID>"
}
is_backing_nsx_managed: true
connected_switch_type {
  value: "N-VDS"
}
 vmk.getLportAttachmentId() value: "<Logical-Port-Attachment-ID>"
 vmk.getLportAttachmentId().getValue() <Logical-Port-Attachment-ID>.
2025-03-10T06:30:31.685Z  INFO NvdsUpgradeTaskExecutor1 NvdsUpgradeReadinessCheckServiceImpl 4858 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] NVDS-UPGRADE exception in precheckConfig <Precheck-ID>: java.util.NoSuchElementException
        at java.util.Collections$EmptyIterator.next(Collections.java:4191)
        at com.vmware.nsx.management.nvdsupgrade.service.NvdsUpgradeReadinessCheckServiceImpl.validateNvdsUpgradeConfig(NvdsUpgradeReadinessCheckServiceImpl.java:1062)
        at com.vmware.nsx.management.nvdsupgrade.service.NvdsUpgradeReadinessCheckServiceImpl.precheckConfig(NvdsUpgradeReadinessCheckServiceImpl.java:919)
        at com.vmware.nsx.management.nvdsupgrade.service.NvdsUpgradeReadinessCheckServiceImpl$NvdsUpgradeTask.run(NvdsUpgradeReadinessCheckServiceImpl.java:1369)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
  • In above example, the transport node that caused the issue can be found using its UUID 
  • vmk ports vmk0 or vmk1 are present in above problematic transport node on the host end but corresponding logical port is missing in NSX.
  • Verify this from NSX UI, navigate to Networking->Logical Switches->Ports.
  • vmk0@NVDS-Overlay@<Transport-Node-UUID> and vmk1@NVDS-Overlay@<Transport-Node-UUID> are not present

Environment

VMware NSX-T Data Center

Cause

This precheck issue arises due to missing logical ports associated with VMkernel adapters (vmks). During the URT precheck, if any vmk on a host is connected to a logical port within a Transport Zone of type OVERLAY, URT attempts to migrate it manually by unprepping the host, indicating that the vmknic is linked to an overlay logical switch.

To verify this, the URT precheck validates all logical ports attached to the vmknics. However, if a vmk’s logical port attachment ID is not associated with any logical port in NSX Manager (MP), the precheck fails to locate the logical port, resulting in a NoSuchElementException. Consequently, since the precheck ID generation fails, the URT status summary reports that the provided precheck ID is invalid.

Resolution

This is a known issue affecting VMware NSX. Follow the steps below to resolve it.


Workaround:

Step 1: Check the Missing VMkernel Port

Run the following command on the affected ESXi host to check if the VMkernel (vmk) port is missing:

# net-stats -l | grep -w vmk1
 

Example output:

xxxxxx 3 0 DvsPortset-3 00:50:56:67:xx:xx vmk1

 

Then, check the Distributed Virtual Switch (DVS) port associated with the VMkernel adapter:

# esxcfg-vmknic -l | grep -w vmk1
 

Example output:

vmk1 38d43aeb-xxxx-xxxx-xxxx-edca8172xxxx IPv4 192.168.x.x 255.255.255.0 192.168.x.x 00:50:56:67:xx:xx 9000 65535 true STATIC vmotion

Step 2: Apply One of the Following Fixes

Approach 1: Migrate VMkernel Adapters Manually

If VMkernel adapters (vmks) don’t need to be migrated under a new CVDS switch:

  1. Open vCenter UI

  2. Navigate to Host > Configure > Virtual Switches

  3. Click the three-dot menu

  4. Select Migrate VMkernel Adapter

  5. Migrate the vmk adapter to a VSS (Virtual Standard Switch) or VDS (Virtual Distributed Switch)

This avoids the original precheck issue.

Approach 2: Restore Missing Logical Ports via API

If using a Transport Node Profile, follow these steps:

  1. Move VMs off the impacted hosts

    • Power off or migrate running VMs from all hosts where the Transport Node Profile is applied.

  2. Locate the Transport Node Profile in NSX UI

    • Go to System > Fabric > Nodes > Host Transport Nodes

    • Find the cluster with the issue

    • Under NSX Configuration, note the Applied Profile name

  3. Find the Transport Node Profile ID

    • Go to System > Fabric > Profiles > Transport Node Profiles

    • Locate the same profile and note its Profile ID

  4. Fetch the Transport Node Profile via API

     
    GET https://<nsx_manager_ip>/api/v1/transport-node-profiles/<transport-node-profile-id>

    • In the response, check if vmk_install_migration exists under the NVDS-Overlay switch.

    • Example payload:

      "vmk_install_migration": [ { "device_name": "vmk1", "destination_network": "38d43aeb-xxxx-xxxx-xxxx-edca8172xxxx" }, { "device_name": "vmk0", "destination_network": "00439af1-xxxx-xxxx-xxxx-22af200dxxxx" } ]
       

  5. If vmk_install_migration is missing, update the Transport Node Profile

    Use this API call:

    PUT https://<nsx_manager_ip>/api/v1/transport-node-profiles/<transport-node-profile-id>
     

    Example payload:

    { "host_switch_spec": { "host_switches": [ { "host_switch_name": "NVDS-Overlay", "vmk_install_migration": [ { "device_name": "vmk1", "destination_network": "38d43aeb-xxxx-xxxx-xxxx-edca8172xxxx" }, { "device_name": "vmk0", "destination_network": "00439af1-xxxx-xxxx-xxxx-22af200dxxxx" } ] } ] } }
     

  6. Verify Logical Ports in NSX Manager

    • Open NSX UI

    • Switch to Manager View

    • Navigate to Networking > Logical Switches > Ports

    • Ensure ports like the following are present:

      • vmk0@NVDS-Overlay@38d43aeb-xxxx-xxxx-xxxx-edca8172xxxx

      • vmk1@NVDS-Overlay@38d43aeb-xxxx-xxxx-xxxx-edca8172xxxx

  7. (If Step 5 was skipped) Reapply the Transport Node Profile

    • Navigate to System > Fabric > Nodes > Host Transport Nodes

    • Select the ESXi cluster

    • Reapply the same Transport Node Profile

  8. Clear Old Precheck ID
    Once logical ports are restored, run this API call to clean up old precheck IDs:

    DELETE https://<nsx_manager_ip>/api/v1/infra/nvds-urt?action=cleanup

    Then, retry generating a new precheck ID.

Approach 3: Fix Individual Transport Nodes

If the issue affects individual Transport Nodes (TN) rather than a cluster, follow one of these steps:

  1. Move VMs off the affected transport node (power off or migrate them).

  2. Apply Approach 2 (API method) but use this endpoint:

    GET https://<nsx_manager_ip>/api/v1/transport-nodes/<transport-node-id>
     

  3. Alternatively, update the Transport Node configuration in NSX Manager:

    • Open NSX UI

    • Navigate to System > Fabric > Nodes

    • Select the affected node

    • Click Actions > Migrate ESXi VMkernel and Physical Adapters

    This triggers Transport Node realization, restoring missing logical ports.

Additional Information

Impact/Risks:

NVDS to VDS migration URT precheck fails


Powered by Wolken Software