VM's Network adapter disconnects with error "Failed to connect virtual device 'Ethernet0'" after migration

Products

• VMware vSphere ESXi
• VMware NSX

Issue/Introduction

Symptoms:

vmware.log shows similar to the following errors:

YYYY-MM-DDTHH:MM:SS| vmx| A100: ConfigDB: Setting ethernet0.allowGuestConnectionControl = "FALSE"
YYYY-MM-DDTHH:MM:SS| vmx| I125: VigorTransport_ServerSendResponse opID=#####-#####-auto-####-##:#########-##-##-##-#### seq=664: Completed EthernetClass request.
YYYY-MM-DDTHH:MM:SS| vmx| I125: VigorTransportProcessClientPayload: opID=#####-#####-auto-####-##:#########-##-##-##-#### seq=665: Receiving Ethernet.ConnectionControl request.
YYYY-MM-DDTHH:MM:SS| vmx| I125: VMXNET3 user: failed to connect 'Ethernet0' to DV Port '1234'.  !!!!
YYYY-MM-DDTHH:MM:SS| vmx| I125: VigorTransport_ServerSendResponse opID=#####-#####-auto-####-##:#########-##-##-##-#### seq=665: Completed Ethernet request with messages.
YYYY-MM-DDTHH:MM:SS| vmx| I125: Msg_Post: Error
YYYY-MM-DDTHH:MM:SS| vmx| I125: [msg.device.badconnect] Failed to connect virtual device 'Ethernet0'.

And there are the following errors reported from vmkernel.log at the same time:

YYYY-MM-DDTHH:MM:SS cpu28:2229027)Net: 2457: connected 1vmtest.eth0 eth0 to vDS, portID 0x100001a
YYYY-MM-DDTHH:MM:SS cpu28:2229027)Net: 3149: associated dvPort 1234 with portID 0x100001a
YYYY-MM-DDTHH:MM:SS cpu28:2229027)DVFilter: 3696: Could not find filter 'vmware-sfw'
YYYY-MM-DDTHH:MM:SS cpu28:2229027)DVFilter: 5768: Failed to add filter vmware-sfw on vNic 0 slot 2: Not found
YYYY-MM-DDTHH:MM:SS cpu28:2229027)WARNING: Net: 3195: DVFilterActivateCommon failed for port 0x100001a: Failure
YYYY-MM-DDTHH:MM:SS cpu28:2229027)Net: 3707: dissociate dvPort 1234 from port 0x100001a
YYYY-MM-DDTHH:MM:SS cpu28:2229027)Net: 3713: disconnected client from port 0x100001a
YYYY-MM-DDTHH:MM:SS cpu23:2099628)LSOM: LSOMCommitDispatch:1614: ce562665-80a0-f5e8-0b28-40f2e91e33cd: duplicate barrier commit at CSN:LSN 4:11513, ignoring barrier commit
YYYY-MM-DDTHH:MM:SS cpu37:2104485)portcfg: PortCfgVsiGetParamPortCfg:392: : not found
YYYY-MM-DDTHH:MM:SS cpu37:2104485)portcfg: PortCfgVsiGetParamPortCfg:392: : not found
YYYY-MM-DDTHH:MM:SS cpu37:2104485)portcfg: PortCfgVsiGetParamPortCfg:392: : not found

And the nsx manager is not in use any more.

Environment

• VMware vSphere ESXi 6.x
• VMware vSphere ESXi 7.x
• VMware vSphere ESXi 8.x

Cause

Stale NSXv VIBs

 

grep -i nsx  commands/localcli_software-vib-list.txt
esx-nsxv                       6.5.0-0.0.8590012                     VMware    VMwareCertified     2021-06-17

Resolution

In order to avoid the same issue in future.

• Navigate to Networking & Security > Installation and Upgrade > Host Preparation. Locate the host on which you upgraded ESXi. The Installation Status or NSX Installation column displays Not Ready.
• Click Not Ready to see more information.
• Select the host and click Actions > Resolve to trigger NSX VIB installation.

Workaround:
In order to trigger nsx vib update,

• Place ESXi host to Maintenance Mode and Exit MM.
• Reboot ESXi Host.

Additional Information

Impact/Risks:
VM network disconnected.