VM loses network connection due to VMXNET3 activation failure
Article ID: 317193
Updated On:
Products
VMware vSphere ESXi
Issue/Introduction
How To Determine Whether an Issue is Related To FreeBSD
Symptoms:
VM loses network connection due to VMXNET3 activation failure
VM's vmware.logs indicate the ethernet device failed to activate -
NIC: connection control message: Failed to connect virtual device 'ethernet0'.
and
2022-06-20T05:08:24.908Z In(05) vcpu-0 - VMXNET3 user: failed to activate 'Ethernet0', status: 0xbad0001
Symptoms:
VM loses network connection due to VMXNET3 activation failure
VM's vmware.logs indicate the ethernet device failed to activate -
NIC: connection control message: Failed to connect virtual device 'ethernet0'.
and
2022-06-20T05:08:24.908Z In(05) vcpu-0 - VMXNET3 user: failed to activate 'Ethernet0', status: 0xbad0001
Environment
VMware vSphere ESXi 7.0.2
VMware vSphere ESXi 7.0.3
VMware vSphere ESXi 7.0.3
Cause
The logs below show evidence that the issue is with the driver used (FreeBSD in this case).
Logs similar to the ones below are seen in var/run/logs/vmkernel.log -
var/run/log/vmkernel.log:37248:2022-06-20T05:08:24.908Z cpu52:12466787)Vmxnet3: 11403: Invalid gen bit for rq: 1, World_Handle: 0x4539b529f000
OR
2021-05-06T13:48:38.504Z cpu43:2148227)Vmxnet3: 11063: Invalid tx completion gen bit in tx queue: 0, world_handle: 0x45396d6a1000
This means that the device activation failure is due to an invalid gen bit in rx/tx queue.
From vmware.log -
2022-06-20T05:07:56.570Z| vcpu-0| I125: VMXNET3 user: Ethernet0 Driver Info: version = 65536 gosBits = 2 gosType = 4, gosVer = 0, gosMisc = 0
2022-06-20T05:07:56.571Z| vcpu-0| I125: VMXNET3 user: Ethernet1 Driver Info: version = 65536 gosBits = 2 gosType = 4, gosVer = 0, gosMisc = 0
2022-06-20T05:07:56.573Z| vcpu-0| I125: VMXNET3 user: Ethernet2 Driver Info: version = 65536 gosBits = 2 gosType = 4, gosVer = 0, gosMisc = 0
gosType is 4 implies that it is FreeBSD OS. FreeBSD has vmxnet3 network driver from the community source and has never made use of the vmxnet3 source code upstreamed to Linux or drivers from VMware.
Logs similar to the ones below are seen in var/run/logs/vmkernel.log -
var/run/log/vmkernel.log:37248:2022-06-20T05:08:24.908Z cpu52:12466787)Vmxnet3: 11403: Invalid gen bit for rq: 1, World_Handle: 0x4539b529f000
OR
2021-05-06T13:48:38.504Z cpu43:2148227)Vmxnet3: 11063: Invalid tx completion gen bit in tx queue: 0, world_handle: 0x45396d6a1000
This means that the device activation failure is due to an invalid gen bit in rx/tx queue.
From vmware.log -
2022-06-20T05:07:56.570Z| vcpu-0| I125: VMXNET3 user: Ethernet0 Driver Info: version = 65536 gosBits = 2 gosType = 4, gosVer = 0, gosMisc = 0
2022-06-20T05:07:56.571Z| vcpu-0| I125: VMXNET3 user: Ethernet1 Driver Info: version = 65536 gosBits = 2 gosType = 4, gosVer = 0, gosMisc = 0
2022-06-20T05:07:56.573Z| vcpu-0| I125: VMXNET3 user: Ethernet2 Driver Info: version = 65536 gosBits = 2 gosType = 4, gosVer = 0, gosMisc = 0
gosType is 4 implies that it is FreeBSD OS. FreeBSD has vmxnet3 network driver from the community source and has never made use of the vmxnet3 source code upstreamed to Linux or drivers from VMware.
Resolution
No resolution
Workaround:
No current workaround. Please collect the live dump and open an SR with support so that we can determine if the above symptoms exist and confirm that it's a FreeBSD issue. Reach out to the FreeBSD team and get the issue fixed.
Workaround:
No current workaround. Please collect the live dump and open an SR with support so that we can determine if the above symptoms exist and confirm that it's a FreeBSD issue. Reach out to the FreeBSD team and get the issue fixed.
Feedback
Yes
No
Powered by
