vNICs are disconnected from VMware NSX segment after VMs fail to migrate

Products

• VMware NSX

Issue/Introduction

The VMs connected on NSX backed segment loses the network connectivity when the HA failover fails to migrate the specific VMs with below symptoms.

• vNICs of such VMs are disconnected after power on, even though "Connect At PowerOn" is enabled.
• It fails to manually connect the vNic again.
• vNICs connected to non-NSX portgroup are not impacted.
• The following entries are seen in the log file location of the corresponding VM /vmfs/volumes/<datastore-name>/<vm-name>/vmware.log

<date/time>| vcpu-0| I005: VMXNET3 user: failed to connect Ethernet1 to network #########-####-####-#######
<date/time>| vcpu-0| I005: Msg_Post: Warning
<date/time>| vcpu-0| I005: [msg.device.startdisconnected] Virtual device 'Ethernet0' will start disconnected.
...
<date/time> In(05) vcpu-0 - [msg.ethernet.vlance.connectFailed] Failed to connect 'ethernet0'.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX

Cause

• The logical port UUID that is assigned to the Virtual Machine's vNic that is connected on the NSX segment is cleared after VM migration fails resulting in the vNICs of the VM to be disconnected.
• Therefore, a failed migration on the destination host results in unregistering the VM and clearing the logical port UUID from the VM. The cleared port state data is recorded in the VM's storage location as shown below

/vmfs/volumes/<datastore-name>/<VM-name>/.dvsData/<DVS-switch-uuid>/<logical-port-uuid>

• If this VM tries to reconnect to an NSX port on the source host again, due to the cleared port UUID, it causes the vNICs to disconnect.

 

Resolution

This is a known issue impacting VMware NSX.

Workaround :-

• Manually connect the disconnected vNICs.
• If it fails, connect the disconnected vNICs to another portgroup and then connect it back to the original VMware NSX segment. It creates new VMware NSX ports and vNICs are attached successfully.

Additional Information

• Ports without external id are cleaned up by a periodic script on ESXi. If the script finds the external id is cleared, it deletes the ports. After the ports are deleted, it fails to manually connect the vNICs again.