ESXi host 6.0 and 6.5 fails with PSOD - "SCSI_DeviceClusteringClearState" when vMotion occurs for a MSCS VM.

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• ESXi 6.0 and 6.5 host fails with a PSOD -  "SCSI_DeviceClusteringClearState" when vMotion occurs for a MSCS VM.
• You see the backtrace similar to:

Backtrace for current CPU #xx, worldID=xyxyxy, fp=0x2005
0xyyyzyyyxyzzy:[0xxxxxxyxxxxxx]SCSI_DeviceClusteringClearState@vmkernel#nover+0x8 stack: 0x0, 0x19be, 0x2005, 0x41802af1f2ed, 0x43074175b0c0
0xyyyzyyyxyyyy:[0xxxzxxxxxxxxx]VSCSI_DestroyDevice@vmkernel#nover+0x2b8 stack: 0x4391da49bf38, 0x2005, 0x4391da49bf30, 0x4315c8417740, 0x41802b517bb8
0xyyyzyyyxyyyx:[0xxxxxxyxxxxxx]UW64VMKSyscallUnpackVSCSIDestroyDev@(user)#<None>+0x10 stack: 0x2005, 0x0, 0xddb7aa4148, 0x742068637573206f, 0x4391da49bfc8
0xyyyzyyxxyyyy:[0xxxxxxxxxzyxx]User_UWVMK64SyscallHandler@(user)#<None>+0x247 stack: 0xddb7aa4148, 0x742068637573206f, 0x4391da49bfc8, 0x2005, 0xddfcbd9a50
0xyxyzyyyxyyyy:[0xxxxxxzyzxxxx]SyscallUWVMK64@vmkernel#nover+0x90 stack: 0x0, 0x0, 0x59, 0xddb7aa4148, 0x2005
0xyyyzyyyxyyyz:[0xxzxxxxxxxxxx]gate_entry_@vmkernel#nover+0x0 stack: 0x0, 0x59, 0xddb7aa4148, 0x2005, 0xddfcbd9a50

• Clustering node virtual machines contain shared non-RDM disk (VMDK or VVOL).
• Clustering node Virtual Machines are in a CAB (Cluster Across Box) configuration.
• SCSI Bus Sharing mode is set to Physical.

Cause

• This issue occurs due to misconfiguration of non-RDM disks across the hosts in physical bus sharing mode during vMotion of MSCS virtual machines. This issue may also occur during vMotion of VVOLs
• PSOD can occur for Oracle RAC VMs too when they are vMotioned

Resolution

This issue is resolved in:
VMware ESXi 6.5, Patch Release ESXi650-201811002. For more information, see https://docs.vmware.com/en/VMware-vSphere/6.5/rn/esxi650-201811002.html.

VMware ESXi 6.0, Patch Release ESXi600-201909001. For more information, see https://docs.vmware.com/en/VMware-vSphere/6.0/rn/esxi600-201909001.html.


Workaround:
To work around this issue, setup the virtual machines cluster by using the following shared storage configuration for MSCS in vSphere:

Note: You must use one of these configuration to setup virtual machines.

• Microsoft Cluster on a Single Host with one or more shared eagerzeroedthick virtual disks.
• Microsoft Cluster on a Single Host with one or more shared RDMs in Virtual Compatibility mode.
• Microsoft Cluster across Hosts with one or more shared RDMs in Physical Compatibility mode.

Note: All these configurations involves sharing of RDMs/VMDKs. Relevant virtual machine Affinity or Anti-affinity rules must be configured to disallow unnecessary virtual machines migration.

For more information, see:

• Setup for Failover Clustering and Microsoft Cluster Service Guide
• Microsoft Cluster Service (MSCS) support on ESXi/ESX (323104)
• Microsoft Windows Server Failover Clustering on VMware vSphere 6.x: Guidelines for supported configurations (315481)