VMs on vVOL datastore become unresponsive or hang with PDL warning

Products

• VMware vSphere ESXi

Issue/Introduction

Shortly after powering on, a VM on a vVOL datastore stops responding and appears to be hung.

The vSphere Client reports:
"The storage backing virtual disk <VM_NAME>_1.vmdk has permanent device loss. You may be able to hot remove this virtual device from the virtual machine and continue after clicking Retry. Click Cancel to terminate this session."

 

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

This issue arises if the PE LUN is unable to process/direct I/O to the subsidiary LUN (i.e. the vVOL). While the PE LUN is not in Permanent Device Loss State (PDL), the second LUNs in this case the vVOLs) effectively are.

This can be confirmed by logging of the SCSI code "H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x68 0x1" in /var/log/vmkernel.log: 

####-##-##T##:##:##.###Z cpu##:2098401)ScsiDeviceIO: 4167: Cmd(0x45d9e17b3148) 0x28, CmdSN 0xdb1 from world ####### to dev "naa.###########################" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x68 0x1  sllid: ###########

 

This sense code is a check condition being reported by the storage. The valid sense data of  0x5 0x68 0x1 = SUBSIDIARY LOGICAL UNIT NOT CONFIGURED

 

Resolution

This is a an array side issue. Please engage your storage vendor support.