ESXi hosts in a vSAN Cluster fail during/after an upgrade

Products

• VMware vSAN
• VMware vSphere ESXi

Issue/Introduction

A PSOD with the following trace is observed at the time or after an upgrade is performed on ESXi hosts that are part of a vSAN cluster.

Environment

vSAN 8.0.2
vSAN 8.0.3

ESXi 8.0.2
ESXi 8.0.3

Cause

This issue is known to occur sporadically and Broadcom Engineering is aware and currently investigating it.

Backtrace includes the following:
PanicvPanicInt@vmkernel
Panic_vPanic@vmkernel
vmk_PanicWithModuleID@vmkernel
SSDLOGFreeLogInt@LSOMCommon
SSDLOG_FreeLogEntry@LSOMCommon
[email protected]
[email protected]
[email protected]
vmkWorldFunc@vmkernel
CpuSched_StartWorld@vmkernel
Debug_IsInitialized@vmkernel
Failed at bora/modules/vmkernel/lsomcommon/ssdlog/ssdopslog.c:735 -- NOT REACHED

Resolution

There is currently no official resolution or patch for this issue.

 

Workaround:

If you encounter this issue, perform the following steps to mitigate it and complete the upgrade:

Enable the tracing parameter: On the remaining ESXi hosts, enable the plogGlobalTracing advanced parameter by running the following command:

esxcfg-advcfg -s 1 /VSAN/plogGlobalTracing

Reboot the host: Manually reboot the ESXi host. The upgrade is expected to complete successfully upon reboot.

Disable the tracing parameter: 
Once the upgrade is fully complete, disable the plogGlobalTracing parameter by running:

esxcfg-advcfg -s 0 /VSAN/plogGlobalTracing

Note: If any of the ESXi hosts crash while the parameter is enabled, please export an ESXi log bundle and engage Broadcom Support for further assistance.