Collecting diagnostic information from an ESX or ESXi host that experiences a purple diagnostic screen

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides instruction about collecting support diagnostic information when troubleshooting a purple screen fault in VMware ESX or ESXi.

Symptoms:

Environment

VMware ESXi 3.5.x Installable
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Installable
VMware ESX Server 3.0.x
VMware ESX 4.1.x
VMware ESX Server 3.5.x
VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.1
VMware ESXi 4.0.x Embedded
VMware ESX 4.0.x
VMware vSphere ESXi 5.5
VMware ESXi 3.5.x Embedded
VMware vSphere ESXi 5.0

Resolution

During a purple diagnostic screen, VMware ESX/ESXi attempts to write a VMkernel coredump to a previously-configured VMKCore (type 0xFC) partition on disk and/or to the ESXi Dump Collector service through the network. For more information, see Configuring an ESX/ESXi host to capture a VMkernel coredump from a purple diagnostic screen (1000328).

Collect three pieces of diagnostic information:

• A screenshot or photograph of the purple diagnostic screen on the console.
• A coredump from the VMkernel placed on disk or sent to the ESXi Dump Collector service.
• A vm-support log bundle from the host following a reboot.

Note: The VMkernel coredump is not included in the vm-support bundle, it must be collected separately from the dump partition or the ESXi Dump Collector host.

To investigate diagnostic information gathered, see Interpreting an ESX host purple diagnostic screen (1004250) or contact VMware Support.

Screenshot or Photograph of the Purple Diagnostic Screen

The purple diagnostic screen on the physical console of the host contains useful information. If possible, the same information is also written to disk and/or . If the storage or network subsystems are experiencing issues, writing a coredump to disk or the network Dump Collector may fail and you may see an error on the purple diagnostic screen similar to:

using slot X of Y... method FAILED. Always capture a screenshot using remote KVM or take a photograph of the physical console's purple diagnostic screen prior to a reboot. This may be the only information available.

Coredump from the VMkernel sent to the ESXi Dump Collector network service

During a purple screen event, information is written to a previously configured ESXi Dump Collector service through the network. If the network subsystem is experiencing issues, writing a coredump through the network may fail and this information may be unavailable. If available, the coredump from the VMkernel includes everything seen on the physical console screen.

Connect to the server where the ESXi Dump Collector service is installed, locate the zdump file from the host, and copy it elsewhere for analysis. The timestamp in the filename corresponds to the time the ESXi Dump Collector service received the coredump, not the time on the ESXi host when the failure occurred.

When a coredump is received by the ESXi Dump Collector service, it logs an entry similar to Dump finished successfully for IPAddress, slot 0, file:/path/to/zdump_ IPAddress_ Date.

• Coredumps received by the VMware ESXi Dump Collector in the vCenter Server 5.x Virtual Appliance
  
  Coredumps received by the ESXi Dump Collector service in the vCenter Server 5.x Virtual Appliance are stored in /var/core/netdumps/. Received files are organized into directories according to the sending hosts' IP address, such as /var/core/netdumps/ 10/11/12/13/zdump_ 10.12.13.14- yyyy-mm-dd-hh_mm-N.
  
  
• Coredumps received by the VMware ESXi Dump Collector 5.x for Windows
  
  Coredumps received by the ESXi Dump Collector service installed on Windows are stored in %ProgramData%\VMware\VMware ESXi Dump Collector\Data\. Received files are organized into directories according to the sending hosts' IP address, such as %ProgramData%\VMware\VMware ESXi Dump Collector\Data\ 10\11\12\13\zdump_ 10.12.13.14- yyyy-mm-dd-hh_mm-N.
  
  
• Coredumps received by the vSphere 4.1 Management Assistant
  
  Coredumps received by the netdumper service in the vSphere 4.1 Management Assistant are stored in /var/core/. Received files are organized into directories according to the sending hosts' IP address, such as /var/core/ 10/11/12/13/zdump_ 10.12.13.14- yyyy-mm-dd-hh_mm-N.
  

Coredump from the VMkernel in a Diagnostic Partition on disk

During a purple screen event, information is written to a previously created and activated VMKcore (type 0xFC) partition on disk. If the storage subsystem is experiencing issues, writing a coredump to disk may fail and this information may be unavailable. If available, the coredump from the VMkernel includes everything seen on the physical console screen.

Use the vSphere Client or vm-support command to copy diagnostic information from the host. For more information, see Collecting diagnostic information for VMware ESX/ESXi (653) or Collecting diagnostic information for VMware ESX/ESXi using the vm-support command (1010705).

• Collecting VMkernel coredumps from ESXi 5.x
  
  During startup of an ESXi 5.x host, the startup script /usr/lib/vmware/vmksummary/log-bootstop.sh checks the defined Dump Partition for new contents. If new content is found, an entry is written to the /var/log/vmksummary.log file citing bootstop: Core dump found.
  
  You can collect logs from an ESXi host either by running vm-support at the command line or by using ExportDiagnostic Data from the vSphere Client. Both methods invoke the vm-support script, which checks the defined Dump Partition for new contents. If new content is found, it is temporarily placed in a vmkernel-zdump file in /var/core/ before being compressed in the vm-support output.
  
  Since the vmkernel-zdump-* coredump file is copied from the Dump Partition while running vm-support, it is not necessary to run vm-support a second time to collect the logs. If vm-support is run multiple times, only the first attempt includes a vmkernel-zdump file.
  
  Note: The directory /var/core/ is often located on a ramdisk, so the vmkernel-zdump files placed within may not persist across a reboot.
  
  It is possible to re-copy the contents of the Dump Partition to a vmkernel-zdump-* coredump file. This may be necessary if the automatically-generated file is deleted. For more information, see Manually regenerating core dump files in VMware ESX and ESXi (1002769).
  
  
• Collecting VMkernel coredumps from ESXi 3.x and 4.x
  
  You can collect logs from an ESXi host either by running vm-support at the command line or by using ExportDiagnostic Data or System Logs from the vSphere Client. Both methods invoke the vm-support script, which checks the defined Dump Partition for new contents. If new content is found, it is placed in a vmkernel-zdump-* file in /var/core/. The filename is generated based on the date: vmkernel-zdump-MMDDYY.HH.mm. The filename generated is written to /tmp/dumppart-copy.NNN.txt.
  
  Since the vmkernel-zdump-* coredump file is copied from the Dump Partition while running vm-support, it is not necessary to run vm-support a second time to collect the logs. The vmkernel-zdump-* file is moved to /var/core/old_cores/ after it is collected. This means that if vm-support is run multiple times, only the first attempt includes a vmkernel-zdump-* coredump file. Running vm-support -a includes the vmkernel-zdump-* coredump files from /var/core/old_cores/ in addition to the normally collected information.
  
  Note: The directory /var/core/ is often located on a ramdisk, so the vmkernel-zdump-* files placed within may not persist across a reboot.
  
  It is possible to re-copy the contents of the Dump Partition to a vmkernel-zdump-* coredump file. This may be necessary if the automatically-generated file is deleted. For more information, see Manually regenerating core dump files in VMware ESX and ESXi (1002769).
  
  
• Collecting VMkernel coredumps ESX 3.x and 4.x
  
  During startup of an ESX 3.x or 4.x host, the startup script /etc/init.d/vmware-late checks the defined Dump Partition for new contents. If new content is found, it is placed in a vmkernel-zdump-* file in /root/. The filename is generated based on the date: vmkernel-zdump-MMDDYY.HH.mm. The filename generated is written to /tmp/dumppart.log. An entry is written to the /var/log/vmksummary log citing "VMkernel error".
  
  You can collect logs from an ESX host either by running vm-support at the command line or by using ExportDiagnostic Data or System Logs from the vSphere Client. Both methods collect all vmkernel-zdump-* files from /root/. The vmkernel-zdump-* file is moved to /root/old_cores/ after it is collected. This means that if vm-support is run multiple times, only the first attempt includes a vmkernel-zdump-* coredump file. Running vm-support -a includes the coredump files from /root/old_cores/ in addition to the normally collected information.
  
  It is possible to re-copy the contents of the Dump Partition to a vmkernel-zdump-* coredump file. This may be necessary if the automatically-generated file is deleted. For more information, see Manually regenerating core dump files in VMware ESX and ESXi (1002769).

Coredump from Service Console

You can collect logs from an ESX host either by running vm-support at the command line or by using ExportDiagnostic Data or System Logs from the vSphere Client. Both methods invoke the vm-support script, which checks the Service Console coredump location (defined in /Misc/CosCoreFile) for a new file. If a new file is found, it is copied and compressed to a file at /root/old_cores/cosNN.core.gz. The original file is deleted.

Since the Service Console coredump file is copied while running vm-support, it is not necessary to run vm-support a second time to collect the logs. The original cosNN.core file is left in /root/old_cores/ after it is collected. This means that if vm-support is run multiple times, only the first attempt includes a Service Console cosNN.core.gz coredump file. Running vm-support -a includes the coredump files from /root/old_cores/ in addition to the normally collected information.

Note: This is not applicable to ESXi, as there is no Service Console.

For more information, see Configuring an ESX host to capture a Service Console coredump (1032962).

Additional Information

By default, the ESXi host vmkernel writes logs to /var/log/messages. These logs can be redirected to an alternate local path or they can be redirected to a remote host. For more information, see the Basic System Administration Guide for your version of ESXi (Embedded or Installable). If you require the support logs beyond the last reboot, it may be advisable to log to both a remote disk and a remote syslogd server.
“vm-support” command in ESX/ESXi to collect diagnostic information
Como coletar informações de diagnóstico de um host ESX ou ESXi que exibe uma tela de diagnóstico roxa
紫色の診断画面が表示される ESX または ESXi ホストから診断情報を収集する
从出现紫色诊断屏幕的 ESX 或 ESXi 主机收集诊断信息
Collecting diagnostic information for VMware ESX/ESXi