NSX Network Outage on ESXi 8.0.3 due to Intel i40en NIC PF Resets

Products

• VMware vSphere ESX 8.x

Issue/Introduction

Virtual Machines (VMs) and NSX Edge nodes residing on specific ESXi 8.0.3 hosts experience a complete loss of network connectivity.
Alarms in NSX Manager indicate transport node or uplink issues.
ICMP ping failures to the Guest OS of affected VMs.
Physical uplinks (vmnic) transition to a 'Down' state simultaneously.

Review the ESXi host logs to confirm the hardware-initiated reset:

Check vmkernel.log: Look for entries indicating a reset requested by the driver:
i40en_SetResetFlags: PF reset requested device resetting

Check vobd.log: Look for correlated uplink transitions:
vob.net.dvport.uplink.transition.down for affected vmnic interfaces.

Environment

Hypervisor: VMware ESXi 8.0.3
Software: VMware NSX-T / VCF
Hardware: Transport nodes utilizing Intel i40en-based Network Interface Cards (NICs)

Cause

The outage is caused by a hardware or driver-level failure of the Intel NICs using the i40en driver. Log analysis indicates that the physical device requests a Physical Function (PF) reset, which triggers a vob.net.dvport.uplink.transition.down event, dropping the physical link and interrupting all North-South and East-West traffic.

 

Resolution

Hardware Diagnostics: Engage the hardware vendor (e.g., Dell) to perform Motherboard/NIC diagnostics and collect vendor-specific logs (e.g., DSET or TSR logs).
Firmware/Driver Alignment: Consult the VMware Compatibility Guide (VCG/HCL) to ensure the current i40en driver and NIC firmware versions are compatible and supported for ESXi 8.0.3.
Updates: If a known firmware defect is identified by the vendor, apply the recommended firmware and driver updates.

Additional Information

ESX-NIC