ESXi host may crash with PSOD (purple screen of death) with the message: BlueScreen: NOT_IMPLEMENTED bora/vmkernel/main/world.c:2307

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• PSOD (purple screen of death) with the message: BlueScreen: NOT_IMPLEMENTED bora/vmkernel/main/world.c:2307
• PSOD (purple screen of death) with the message: @BlueScreen: NMI IPI: Panic requested by another PCPU. RIPOFF(base):RBP:CS
• ALERT in the vmkernel.log similar to:

    ALERT: Heap: 2746: Unable to complete wait for non-empty heap     (worldGroup.2611820): Timeout

• PSOD (purple screen of death) backtrace for this issue will look like the following:

Backtrace for current CPU #8, worldID=2097296, fp=0x0
0x45390391bc00:[0x420038114d31]PanicvPanicInt@vmkernel#nover+0x1f5 stack: 0x100, 0x420038114d31, 0x0, 0x420000000001, 0x420038114d31
0x45390391bcb0:[0x4200381153a0]Panic_NoSave@vmkernel#nover+0x4d stack: 0x45390391bd10, 0x45390391bcd0, 0x420038427e15, 0x42003879d696, 0x8f6
0x45390391bd10:[0x4200381158ad]Panic_OnAssertAt@vmkernel#nover+0xba stack: 0x8f600000000, 0x42003879d696, 0x42003878ba99, 0x42003879d287, 0x4200387a8f5e
0x45390391bd90:[0x42003816855f]Int6_UD2Assert@vmkernel#nover+0x260 stack: 0x0, 0x0, 0x0, 0x420038161068, 0x0
0x45390391bdc0:[0x420038161067]gate_entry@vmkernel#nover+0x68 stack: 0x0, 0xbad0021, 0x420041000208, 0xbad0003, 0x1
0x45390391be80:[0x420038147136]World_DestroyHeap@vmkernel#nover+0x4e stack: 0x43167da00000, 0x420038147252, 0x4, 0x4200387dcb20, 0x45390391bef0
0x45390391bea0:[0x420038147251]WorldGroupCleanup@vmkernel#nover+0xe6 stack: 0x45390391bef0, 0x4200380f1def, 0x4301aa401234, 0x4301a8001220, 0x431149601220
0x45390391bec0:[0x4200380f1dee]InitTable_Cleanup@vmkernel#nover+0x27 stack: 0x431149601220, 0x42003814cd47, 0xffffffffffffffff, 0x4301aa201678, 0x45391579f000
0x45390391bee0:[0x42003814cd46]World_TryReap@vmkernel#nover+0x3d3 stack: 0x45391579f000, 0x45390391bf00, 0x0, 0x0, 0x0
0x45390391bfa0:[0x420038117582]ReaperWorkerWorld@vmkernel#nover+0xaf stack: 0x45390389f100, 0x45390391f100, 0x0, 0x0, 0x0
0x45390391bfe0:[0x420038428eca]CpuSched_StartWorld@vmkernel#nover+0x7b stack: 0x0, 0x4200380d788c, 0x0, 0x0, 0x0
0x45390391c000:[0x4200380d788b]Debug_IsInitialized@vmkernel#nover+0xc stack: 0x0, 0x0, 0x0, 0x0, 0x0
VMware ESXi 8.0.0 [Releasebuild-20513097 x86_64] NOT_IMPLEMENTED bora/vmkernel/main/world.c:2307

Note: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 8.0.x
VMware vSphere ESXi 7.0.x

Cause

In a rare condition, vmkpollcontext object leak occurs in the vmkernel which eventually results in a PSOD.

iLO interrupt storm.

Resolution

Any HPE Gen10 / Gen10 Plus / Gen11 platforms running VMware ESXi 7.0 and ESXi 8.0 (or later) May Experience a PSOD With a "BlueScreen: NOT_IMPLEMENTED bora/vmkernel/main/world.c:2307" Message.

 

This issue is fixed by updating the HPE iLO Native driver component to v10.8.2 (or later).

 

Download link:

HPE iLO Native Driver for ESXi 7.0 Version 10.8.2 (or later)

 

Reference Advisory link from HPE for this PSOD :

 

https://support.hpe.com/hpesc/public/docDisplay?docId=a00138303en_us

Additional Information

Impact/Risks:

• No performance impact is associated with the workaround.
• Hardware related events can be monitored through ILO or external monitoring software if any, vCenter/ESXi will not get any hardware health related information.

If the workaround is being applied, additional applications will need to be removed (HPE AMS/iSUT/iLORest). The removal of these applications will have the following impact due to compensatory effects.

• OS upgrade/SW/FW updates via VMware vLCM-HSM method will be impacted.
• HPE SPP/SSP online update method will be impacted.
• Losing information from ESXi in the iLO, this impact's HPE system management.
• Customer may lose automation ability for jobs/scripts which needs the information from AMS (SNMP) and iLORest.

Note: These impacts need to be evaluated before any action is taken.