ESXi 7.x host fails with purple diagnostic screen with "#PF Exception 14 in world xxxxx:vmx IP..." in NSX-T Environment

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• The purple diagnostic screen contains entries similar to:

2022-11-08T16:33:19.982Z cpu62:2118052)@BlueScreen: #PF Exception 14 in world xxxx:vmx IP 0x420035093b78 addr 0x8
PTEs:0x15504d5027;0x1307376027;0x0;
2022-11-08T16:33:19.996Z cpu62:2118052)Code start: 0x420034400000 VMK uptime: 0:00:11:29.695
2022-11-08T16:33:20.024Z cpu62:2118052)0x4539a569bd00:[0x420035093b78][email protected]#v2_9_0_0+0x1b0 stack: 0x43114946f650
2022-11-08T16:33:20.054Z cpu62:2118052)0x4539a569bd70:[0x420035093d41][email protected]#v2_9_0_0+0x16a stack: 0x465e80001220
2022-11-08T16:33:20.077Z cpu62:2118052)0x4539a569bdc0:[0x420035fdb12c]FSR_CheckProgress@esx#nover+0x1e5 stack: 0x4539a0d9f000
2022-11-08T16:33:20.101Z cpu62:2118052)0x4539a569be40:[0x420035fa5b82]Migrate_CheckProgress@esx#nover+0x67 stack: 0x432effa09280
2022-11-08T16:33:20.128Z cpu62:2118052)0x4539a569be70:[0x42003492a8ee]UW64VMKSyscallUnpackMigrateCheckProgress@vmkernel#nover+0x1f stack: 0x0
2022-11-08T16:33:20.154Z cpu62:2118052)0x4539a569bee0:[0x4200348b172a]User_UWVMK64SyscallHandler@vmkernel#nover+0x183 stack: 0x29036169
2022-11-08T16:33:20.174Z cpu62:2118052)0x4539a569bf40:[0x42003454af78]SyscallUWVMK64@vmkernel#nover+0x90 stack: 0x0
 

• This issue is observed during vMotion of virtual machines in NSX-T Environment
• Before the crash, in the /var/run/log/vmkernel.log, entries may be found similar to:

2022-11-08T16:33:19.766Z cpu62:2118052)Received vMotion aborted signal...
2022-11-08T16:33:19.766Z cpu62:2118052)Save state called for filter nic-2113840-eth231-vmware-sfw.1
2022-11-08T16:33:19.766Z cpu62:2118052)Received vMotion aborted signal...
2022-11-08T16:33:19.766Z cpu62:2118052)Save state called for filter nic-2113840-eth232-vmware-si.12

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on the environment.

Environment

VMware vSphere ESXi 7.x

Cause

This issue occurs due to a crash in DVFilter module during vMotion of Virtual Machines.

Resolution

This issue is resolved in ESXi 7.0 Update 3i build 20842708 available through the Broadcom Support Portal. However, it's advised to update to the latest ESXi 7.0 release. Please find the vCenter Server download packages by doing the following:

1. Login to the Broadcom Support Portal
2. Select "My Downloads"
3. Select "VMware vSphere"
4. Select the entitlement, i.e,
    "VMware vSphere - Enterprise Plus"
5. This window defaults to "Primary Downloads". Select the major version, i.e.
    "7.0" 
6. In the row titled "VMware vSphere Hypervisor (ESXi)" select "View Group"
7. In the upper right of the next window use the pull-down menu to select the minor version, i.e,
    "7.0U3n"
8. Select the cloud-download icon to download the ESXi ISO image or the ESXi offline bundle format file

Note: If looking for a vendor's custom ISO file, in step 5 select "Custom ISOs" toward the top instead