Error: Machine Check Exception (MCE) resulting in a Purple Screen of Death (PSOD)
Article ID: 372284
Updated On:
Products
Issue/Introduction
An ESXi host encountering a Purple Screen of Death (PSOD) might display a reference to a "Machine Check Exception".
The ESXi host may enter a "Not Responding" state in vCenter Server.
At the ESXi console, the PSOD screen will have entries similar to the following:
VMware ESXi #.#.# [Releasebuild-######## x86_64]Machine Check Exception on PCPU## in world ######:idle51System has encountered a Hardware Error - Please contact the hardware vendorUncorrectable/recoverable memory error in world ####; unable to recover in kernel contextData Cache DataRead ErrorThe purple screen backtrace may also include vSAN-related functions such as
BucketlistInsertInNode,Bucketlist_Insert,Rangemap_Insert, orPLOGPrepareWrite.The following entries are observed in the /var/run/log/vmkernel.log file on the ESXi host:
YYYY-MM-DDTHH:MM:SS.FFFZ cpu##:40848027)ALERT: MCA: 200: SRAR Excp G7 B1 ###### Cache Hierarchy: Level 0 Data Cache DataRead Error.YYYY-MM-DDTHH:MM:SS.FFFZ cpu##:40848027)MCAIntel: 1120: Force retiring MPN ###### to recover from MCA error detected by cpu## in ####.Machine Check Exception on PCPU## in world 2099051:VSAN_0x4##18The error can also be caused by a failing hardware device. In such cases, the PSOD screen may report an error similar to the following:
YYYY-MM-DDTHH:MM:SS.FFFZ cpu##:########)IDT: ### : Uncorrectable/unrecoverable machine check errorYYYY-MM-DDTHH:MM:SS.FFFZ cpu##:########)MCA: ### : UC Excp G4 86 Sbb###########e#b AB M###### P8/8 I/O error reported by PCI ####:##:##.#.
Environment
- VMware vSphere ESXi 7.x / 8.x / 9.x
- VMware Cloud Foundation 4.x / 5.x / 9.x
- Cisco UCS, Dell PowerEdge, HPE ProLiant Servers
Cause
The Machine Check Architecture (MCA) is a CPU feature designed to detect and report hardware anomalies. When the hardware detects a critical or fatal condition, it raises a Machine Check Exception (MCE). These exceptions are considered severe and unrecoverable, resulting in an expected ESXi host crash, often resulting in a Purple Screen of Death (PSOD).
In this scenario, the MCE was categorized as a Software Recoverable Action Required (SRAR), which indicates the following:
- Uncorrectable: The error cannot be automatically corrected by hardware.
- Recoverable: A system-level action could theoretically mitigate the issue.
- Action Required: Specific corrective actions, such as terminating the thread accessing the affected Memory Page Number (MPN), are required.
The faulty thread was executing within a critical vmkernel context (e.g., a vSAN thread like VSAN_0x45018) and the ESXi host was unable to isolate or terminate it, leaving the hypervisor with no viable recovery path. As a result, the MCE was escalated to a fatal system error, causing the ESXi host to halt to ensure data integrity.
Resolution
The underlying hardware failure must be addressed. Perform the following steps:
- Reboot: Restart the host via the hardware management interface (e.g., iDRAC, ILO, CIMC) to attempt temporary service restoration.
- Review Management Logs: Inspect the hardware management interface for indicators such as:
- Predictive failures on memory modules (DIMMs).
- CPU internal errors or bank errors (e.g., uncorrectable memory errors).
- PCI parity or bus errors.
- Run Diagnostics: Execute a comprehensive hardware diagnostic test (Vendor-specific) to identify the faulty physical component (CPU or Memory).
- Engage Hardware Vendor: Contact the server OEM (e.g., Cisco, Dell, HPE). Provide the PSOD screenshot and exported System Event Logs (SEL/IPMI) for component replacement.
- Update BIOS/Firmware: Verify the host is compliant with the VMware Compatibility Guide. Apply missing BIOS updates to resolve known microcode stability issues that can trigger MCEs.
To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.
Additional Information
Feedback
