"Failed at bora/modules/vmkernel/virsto/space_mgmt/space_mgmt_meta_page.c" PSOD on vSAN Host

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• Affected vSAN Hosts running: ESXi 6.5.x , 6.6.x, 6.7.x and potentially ESXi 6.0

• vSAN Host may hit a PSOD due to a bad capacity tier disk.

• The Host will not boot and all the disk groups on that Host will not be available.

Environment

VMware vSAN 6.x

VMware vSAN 7.x

Cause

• vSAN is trying to deallocate a block for a vDisk or Snapshot  on the virsto filesystem against a Bad Disk offsets.

• It may also be caused by double free allocation on affected disk group. 

2019-06-11T17:25:01.132Z cpu18:67282)@BlueScreen: Failed at bora/modules/vmkernel/virsto/space_mgmt/space_mgmt_meta_page.c:480 -- VMK_ASSERT(vmk_BitVectorTest(&a_bm_p->bitMap, blk_n))
2019-06-11T17:25:01.132Z cpu18:67282)Code start: 0x418004e00000 VMK uptime: 0:00:40:36.953
2019-06-11T17:25:01.132Z cpu18:67282)0x43913691b810:[0x418004eed491]PanicvPanicInt@vmkernel#nover+0x545 stack: 0x418004eed491
2019-06-11T17:25:01.132Z cpu18:67282)0x43913691b8b0:[0x418004eed827]Panic_vPanic@vmkernel#nover+0x23 stack: 0x430810efba98
2019-06-11T17:25:01.133Z cpu18:67282)0x43913691b8d0:[0x418004f01461]vmk_PanicWithModuleID@vmkernel#nover+0x41 stack: 0x43913691b930
2019-06-11T17:25:01.133Z cpu18:67282)0x43913691b930:[0x418006191ef9][email protected]#0.0.0.1+0x77d stack: 0x430811235ec6
2019-06-11T17:25:01.133Z cpu18:67282)0x43913691b9b0:[0x418006192512][email protected]#0.0.0.1+0x346 stack: 0x430810efbc38
2019-06-11T17:25:01.134Z cpu18:67282)0x43913691ba20:[0x41800618c43c][email protected]#0.0.0.1+0xc stack: 0x430815441470
2019-06-11T17:25:01.134Z cpu18:67282)0x43913691ba30:[0x4180061aced0][email protected]#0.0.0.1+0xb8 stack: 0x430815440130
2019-06-11T17:25:01.134Z cpu18:67282)0x43913691bad0:[0x4180061aef0f][email protected]#0.0.0.1+0x23b stack: 0xe00000002
2019-06-11T17:25:01.135Z cpu18:67282)0x43913691bb60:[0x4180061b0688][email protected]#0.0.0.1+0x240 stack: 0x4018
2019-06-11T17:25:01.135Z cpu18:67282)0x43913691bc40:[0x4180061616a6][email protected]#0.0.0.1+0x2fe stack: 0x12
2019-06-11T17:25:01.135Z cpu18:67282)0x43913691bd40:[0x418006220bb6][email protected]#0.0.0.1+0x146 stack: 0x2412
2019-06-11T17:25:01.136Z cpu18:67282)0x43913691bdf0:[0x4180061f5946][email protected]#0.0.0.1+0x1aa stack: 0x2001
2019-06-11T17:25:01.136Z cpu18:67282)0x43913691bed0:[0x41800600aef2][email protected]#0.0.0.1+0x51a stack: 0x430995f1a8e8

Resolution

The issue is fixed in 6.0 U3 (5050593), vSAN 6.5 P04 (6765664) and VSAN 7.0 U1.

NOTCIE: There is a chance that this PSOD is causing a DU/DL (Data Unavailable/Data Loss) situation. Do not recreate, or delete Disk group, or wipe the Partition table on the Host as the first step, as this may cause a DL situation for few or more VMs.

If the above symptoms and issue matches in the latest builds, please contact Broadcom Support to investigate the issue.