VMs connected to NSX segments are unable to reconnect after a powered off/on event
search cancel

VMs connected to NSX segments are unable to reconnect after a powered off/on event

book

Article ID: 314276

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • You are using VMware NSX-T 3.2.X.
  • After a VM power off/on event, the VM loses connectivity.
  • Checking the network for the VM under "edit settings" in vCenter illustrates the network as disconnected.
  • Attempting to connect the network fails with error: Failed to connect virtual device ethernetX.
  • Attempting to vMotion the VM fails with error: The object or item referred to could not be found. vDS dvs port cannot be found
  • Checking on the ESXi support bundle in /var/run/log/hostd.log we can see the following error:
2022-11-30T15:24:14.847Z info hostd[2102630] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=########-########-auto-9sjt3-h5:72088509-##-##-##-##-01-f7-f5f5] Event 1401 : Error message on ########
up on #######.### in ha-datacenter: Failed to connect virtual device 'Ethernet0'.

 
  • Checking on the NSX Manager support bundle in /var/log/proton/nsxapi.log we can see the following errors
2022-11-30T15:14:10.817Z WARN L2TaskExecutor11 LogicalPortServiceImpl 4882 SWITCHING [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Failed to process vif attach for logicalSwitchUuid 30497487-####-####-####-####-47a599e757d3
...
2022-11-30T15:14:10.817Z WARN L2TaskExecutor11 LogicalPortServiceImpl 4882 SWITCHING [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Stack trace java.lang.UnsupportedOperationException
...
2022-11-30T15:14:10.817Z INFO L2TaskExecutor11 TraceAspectAjIf 4882 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] abnormally finish tracing a workflow with name addVnicAttachmentImpl

 
 

NOTE: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware NSX-T Data Center 3.x
VMware NSX 4.x

Cause

This is caused by calling to remove on iterator, which results in a failed to attach vif for logical segment.
 
 

Resolution

This is resolved in VMware NSX-T 3.2.3 and NSX 4.1.1.

For VCF environments, please use the Async Patch Tool to upgrade to NSX-T 3.2.3
Additional Information can be found in Applying individual product updates to VMware Cloud Foundation environments using Async Patch Tool (AP Tool).


Workaround:
Connect the identified virtual machine to a vCenter portgroup and connect back to target NSX segment.

If the issue persists after trying this workaround, please Broadcom Support and reference this KB.