Migration from NSX-V to NSX-T fails leaving VMs inconsistently distributed across OrgVDCs

Products

VMware Cloud Director

Issue/Introduction

During an NSX-V to NSX-T migration within VMware Cloud Director (VCD) using the Migration Coordinator or NSX Migration for Cloud Director tool, the process may fail during the "Migrate Workloads" phase.

Symptoms:

VMs are "split" between the source NSX-V OrgVDC and the target NSX-T OrgVDC.
Some VMs may appear as "Inconsistent" or "Powered Off" in the VCD UI.
Migration logs (vcd-ni-migrator.log) show failures during the moveVApp or migrateVApp API calls.
Network connectivity is lost for VMs that were physically moved to NSX-T hosts but failed the logical metadata update in VCD.

Environment

10.x

Cause

The failure is caused by an interruption in the atomicity of the workload migration task. This occurs when the physical vMotion of the VM completes, but the subsequent VCD API calls to update the VM's network backing and OrgVDC association fail due to timeouts, database locks, or unexpected API responses (e.g., HTTP 500). This leaves the VM in a "zombie" state where its physical location (vCenter) and logical metadata (VCD) are mismatched.

Resolution

To recover services, the VMs must be manually reconciled back to the source OrgVDC before a re-attempt can be made.

Identify Affected VMs: Compare the VM inventory in the source and target OrgVDCs to identify "orphaned" or mismatched VMs.
Revert VM Backing (vCenter):
- If the VM was moved to an NSX-T host/datastore, use vMotion to migrate the VM back to the source NSX-V cluster and datastore.
- Disconnect the VM network adapter and reconnect it to the original NSX-V backed Port Group.
Re-register VM in VCD (Manual Recovery):
- This process strips the "Cloud" metadata from the VM at the vCenter level, allowing it to be treated as a new "orphaned" object that can be cleanly imported back into the original VCD VApp.
  
  1. Prepare the VM in vSphere
  1. Power Off the affected Virtual Machine.
  2. Right-click the VM in the vSphere Client and select Edit Settings.
  3. Navigate to VM Options > Advanced > Edit Configuration.
  4. Locate the parameter cloud.uuid.
  5. Remove the value or the entire row for cloud.uuid.
    - Note: This dissociates the VM from its previous failed VCD migration state.
  6. Right-click the VM and select Remove from Inventory.
    - Caution: Do NOT select "Delete from Disk."
  2. Re-register and Align Placement
  1. Browse the Datastore where the VM files reside.
  2. Locate the .vmx file, right-click it, and select Register VM.
  3. Target the specific NSX-V Prepared Cluster and Resource Pool associated with the source OrgVDC.
  4. Ensure the VM's Network Adapter is mapped to the correct NSX-V Logical Switch (Standard/Distributed Port Group).
  3. Import back into VMware Cloud Director
  1. Log in to the VCD Provider Portal or Tenant Portal.
  2. Navigate to the Library or Data Centers > VApps.
  3. Open the Source VApp and select Actions > Add VM (or use the "Import from vCenter" function).
  4. Select the vCenter Server and locate the VM you just re-registered.
  5. Complete the wizard to import the VM into the VApp. VCD will generate a new cloud.uuid and re-establish management control.
  6. Power On the VM and verify guest customization and network heartbeat.
Cleanup Target Metadata:
- Delete any partially migrated vApp templates or "dummy" network objects in the target NSX-T OrgVDC created by the migration tool.
Verify Connectivity: Power on the VMs in the source OrgVDC and confirm network reachability.
Retry Migration: Once the environment is clean, initiate the migration again, ensuring all prerequisites (L2 Bridging health and API timeouts) are verified.