ESXi crashes with PSOD PF Exception 14 in world, Module(s) involved in panic [vvol Version 1.0]

Products

• VMware vSphere ESXi

Issue/Introduction

When performing svmotion, vmotion, snapshot, cloning ect.

ESXi may crash with PSOD similar to following:

0x000042000c57dda6 in VvolResetTrackingRangeInt (args=0x43233fe8db50, vvolDevIn=0x43233fe3d840) at bora/modules/vmkernel/vvol/vvol_module.c:5730
VVolAllocationTracking (vvolDevIn=0x43233fe3d840, args=0x43233fe8db50) at bora/modules/vmkernel/vvol/vvol_module.c:5854
VVolIoctlHandler (vvolDev=vvolDev@entry=0x43233fe3d840, args=args@entry=0x4327d941e170) at bora/modules/vmkernel/vvol/vvol_module.c:6580
0x000042000c57f651 in VVolIoctl (fdsHandleID=73818674634656, cmd=<optimized out>, dataInOut=0x4327d941e170) at bora/modules/vmkernel/vvol/vvol_module.c:7169
0x000042000ac6ca3f in FDS_Ioctl (dataInOut=0x4327d941e170, cmd=FDS_IOCTL_VVOL, fdsHandle=<optimized out>) at bora/vmkernel/private/fsDeviceSwitch.h:779
DevFSIoctl (fileDesc=0x430cf21074c0, fhID=7543947, cmd=IOCTLCMD_VMFS_VVOL, dataIn=0x4327d941e170, result=0x4327d941e190) at bora/vmkernel/filesystems/devfs/devfs.c:5755
0x000042000ac44af0 in FSSVec_Ioctl (desc=<optimized out>, fhID=<optimized out>, cmd=<optimized out>, dataIn=<optimized out>, result=<optimized out>) at bora/vmkernel/filesystems/fsSwitchVec.c:778
0x000042000ac40dc7 in FSSObjectIoctlCommon (fhID=7543947, file=0x430cf21074c0, cmd=IOCTLCMD_VMFS_VVOL, dataIn=0x4327d941e170, result=0x4327d941e190, ioFlags=<optimized out>)
    at bora/vmkernel/filesystems/fsSwitch.c:4780
0x000042000ac40cb8 in FSS_IoctlByFH (fileHandleID=7543947, cmd=cmd@entry=IOCTLCMD_VMFS_VVOL, dataIn=dataIn@entry=0x4327d941e170, result=result@entry=0x4327d941e190,
    ioFlags=ioFlags@entry=FS_INVALID_FLAG) at bora/vmkernel/filesystems/fsSwitch.c:4876
0x000042000b1d39e9 in UserFileVMFSIoctl (vmfsObj=0x45d9c1958ce0, userArg=802446965536, cmd=<optimized out>) at bora/vmkernel/user/userFile.c:1569
UserFileIoctl (vmfsObj=0x45d9c1958ce0, cmd=<optimized out>, userArg=802446965536, ioctlReturnCode=<optimized out>) at bora/vmkernel/user/userFile.c:2028
0x000042000b1a769c in UserVmfs_Ioctl (obj=<optimized out>, cmd=<optimized out>, userArg=802446965536, ioctlReturnCode=<optimized out>) at bora/vmkernel/user/userVmfs.c:1847
0x000042000b1d8a72 in LinuxFileDesc_Ioctl (fd=<optimized out>, cmd=3059, userData=802446965536) at bora/vmkernel/user/linuxFileDesc.c:4719
0x000042000b1836c2 in User_LinuxSyscallHandler (fullFrame=0x4539eef9bf40) at bora/vmkernel/user/user.c:2104
0x000042000ad670b7 in gate_entry ()
0x000000bac60da9c7 in ?? ()

Environment

ESXi 8x
ESXi 7x

Cause

This issue is caused by a rare race condition.

The Bitmap file should be locked while doing a reset, storage migration or any workflow which is reading the allocation map file for the disk, race in vmkernel which reads the tracking and clears the tracking file simultaneously causes invalid memory access which in turn triggers the PSOD.

Resolution

Issues is fixed in ESXi 8.0.3