PSOD on ESXi Host with "VERIFY bora/vmkernel/sched/cpusched"
search cancel

PSOD on ESXi Host with "VERIFY bora/vmkernel/sched/cpusched"

book

Article ID: 415967

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • An ESXi host experienced a Purple Screen of Death (PSOD) with the error "VERIFY bora/vmkernel/sched/cpusched". This error indicates a critical system failure within the CPU scheduler component of the ESXi kernel, leading to an unexpected shutdown of the host. This document outlines the details of the PSOD event and provides steps for investigation and potential resolution.
  • VERIFY bora/vmkernel/sched/cpusched.c:11152
    cr0=0x8001003d cr2=0x4507000 cr3=0x607000 cr4=0x10216c
    FMS=06/6a/6 uCode=0xd0003f5
    *PCPU8:2098575/vmk1-rx-2
    PCPU 0: UUVUVSUSSVUVSSUVSIVUVSVUVVVVVVSVVV IVIVIVIVVSVIVVVSUVUUVUSUUUUUUS
    PCPU 64: VUVUVUVVVSVSIIVIIIVVVIVISVVIVVSI
    Code start: 0x420008200000 VMK uptime: 96:04:32:27.705
    0x4539a5c9bb50: [0x4200082ff 107]PanicvPanic Int@vmkernel#nover+0x327 stack: 0x4539a5c9bc28
    0x4539a5c9bc20 : [0x4200082ff660]Panic_NoSave@vmkernel#nover+0x4d stack: 0x4539a5c9bc80
    0x4539a5c9bc80: [0x4200082ffbf1]Panic_OnAssertAt@vmkernel#nover+Oxba stack: 0x2b9000000000
    0x4539a5c9bd00: [0x420008355716]Int6_UD2Assert@vmkernel#nover+0x27f stack: 0x0
    0x4539a5c9bd30 : [0x42000834e067]gate_entry@vmkernel#nover+0x68 stack: 0x0
    0x4539a5c9bdf0: [0x4200085b4174]CpuSchedWait@vmkernel#nover+0x2e4 stack: 0x0
    0x4539a5c9bf70: [0x4200085b4234]CpuSched_NoEvqWait@vmkernel#nover+0x19 stack: 0x0
    0x4539a5c9bf80 : [0x4200096dd224]TcpipDispatchWor1d@(tcpip4)#<None>+0x225 stack: 0x0
    0x4539a5c9bfe0: [0x4200085b4d55]CpuSched_StartWor1d@vmkernel#nover+0x86 stack: 0x0
    0x4539a5c9c000: [0x4200082c4ddf ]Debug_Is Initialized@vmkernel#nover+0xc stack: 0x0
    base fs=0x0 gs=0x420042000000 Kgs=0x0
    YYYY-MM-DDTHH:MM:SS cpu65:2107268)BC: 3177: File host-####-hb closed with dirty buffers. Possible data loss.
    YYYY-MM-DDTHH:MM:SS cpu48:2107284)BC: 3177: File host-####-hb closed with dirty buffers. Possible data loss.
    No place on disk to dump data.
    Finalized dump header (15/15) FileDump: Successful.
    No port for remote debugger. "Escape" for local debugger.

Environment

  • VMware vSphere ESXi 7.x

Cause

The PSOD with the "VERIFY bora/vmkernel/sched/cpusched" error could be attributed to a couple of suspected causes:

  • CPU State Mismatch: This issue might stem from a CPU state mismatch. If this is the case, it is recommended to engage the hardware vendor. Potential solutions could involve downgrading the firmware or installing a new driver.
  • Spin Lock and Unlock Mismatch: Another possibility is a spin lock and unlock mismatch within the software path. 

Resolution

  • Broadcom suggests downgrading the NMLX firmware. If the PSOD re-occurs, please reach out to Broadcom support for further investigation.
Powered by Wolken Software