VMs connected to NSX segments are unable to reconnect after a powered off/on event
book
Article ID: 314276
calendar_today
Updated On:
Products
VMware NSX
Issue/Introduction
You are using VMware NSX-T 3.2.X.
After a VM power off/on event, the VM loses connectivity.
Checking the network for the VM under "edit settings" in vCenter illustrates the network as disconnected.
Attempting to connect the network fails with error: “Failed to connect virtual device ethernetX.”
Attempting to vMotion the VM fails with error: “The object or item referred to could not be found. vDS dvs port cannot be found”
Checking on the ESXi support bundle in/var/run/log/hostd.log we can see the following error:
2022-11-30T15:24:14.847Z info hostd[2102630] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=########-########-auto-9sjt3-h5:72088509-##-##-##-##-01-f7-f5f5] Event 1401 : Error message on ######## up on #######.### in ha-datacenter: Failed to connect virtual device 'Ethernet0'.
Checking on the NSX Manager support bundle in /var/log/proton/nsxapi.log we can see the following errors
2022-11-30T15:14:10.817Z WARN L2TaskExecutor11 LogicalPortServiceImpl 4882 SWITCHING [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Failed to process vif attach for logicalSwitchUuid 30497487-####-####-####-####-47a599e757d3
... 2022-11-30T15:14:10.817Z WARN L2TaskExecutor11 LogicalPortServiceImpl 4882 SWITCHING [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Stack trace java.lang.UnsupportedOperationException ... 2022-11-30T15:14:10.817Z INFO L2TaskExecutor11 TraceAspectAjIf 4882 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] abnormally finish tracing a workflow with name addVnicAttachmentImpl
NOTE: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.
Environment
VMware NSX-T Data Center 3.x VMware NSX 4.x
Cause
This is caused by calling to remove on iterator, which results in a failed to attach vif for logical segment.
Resolution
This is resolved in VMware NSX-T 3.2.3 and NSX 4.1.1.