Performance Data Collection using esxtop and resxtop

Products

VMware vSphere ESXi

Issue/Introduction

This article explains how to use performance collection tools to gather data for analysis of faults such as:

Unresponsive ESX hosts.
Unresponsive virtual machines.
ESXi hosts purple diagnostic screens.

Environment

VMware vSphere ESXi 6.x

VMware vSphere ESXi 7.x

VMware vSphere ESXi 8.x

Resolution

Why gather performance data for a fault?

If the diagnostic logs do not help you determine the cause of a fault, you may need to use performance collection tools to gather further data for analysis.

Set up performance collection tools to gather data about faults that may occur.

Performance gathering tools

VMware recommends the following tools for gathering performance data:

esxtop

The esxtop tool provides performance statistics for the entire ESX/ESXi host. It provides details of network, storage, CPU, and memory load from the VMkernel perspective. It provides details on a VMkernel world basis.

To collect the data for a longer period, run esxtop in batch mode. Direct the output to a file to be reviewed after the fault.

To run the esxtop tool, run the command:

# esxtop –b –d <delay in seconds> [-n <iterations>] > output-perf-statistics-file.csv

Note:

Make sure you have enough free space available on the location from where you are running the command and on the datastore where the output file saved. Also make sure there is no performance issue on the datastore where the output file is saved.

resxtop

Like esxtop, the resxtop tool provides performance statistics for a specified ESX host in the network. It provides the same performance information as esxtop and may be used either after deploying the VMware vSphere Management Assistant (vMA) virtual appliance or installing the VMware Command-Line Interface (vCLI).

resxtop is particularly helpful for ESX and ESXi deployments where an interactive shell login is unavailable.

To run the resxtop tool and collect batch performance data, log into the vMA or open the vCLI, and execute the command:

# resxtop [server] [vihost] [portnumber] [username] -b -d <delay in seconds> [-n <interations>] > output-perf-statistics-file.csv

vm-support -s

Use the vm-support command with the -s parameter to collect performance statistics, system configuration information, and logging. Submit the file generated by this command to VMware Support for further assistance, if required.

For information on the vm-support command, see Collecting performance snapshots using vm-support

Performance Monitor (PERFMON.EXE)

Microsoft's Performance Monitor is a utility that comes with every Microsoft Windows NT-based Operating System. This utility can be used to monitor local and remote Microsoft Windows machines. It can log performance data and display data from logs or real-time data.

This utility is useful when reviewing data collected from the esxtop tool and for troubleshooting virtual machine unresponsiveness. When using Performance Monitor for virtual machine unresponsiveness, collect the data remotely from another Microsoft Windows machine so that the utility does not affect the data being gathered.

For more information about Performance Monitor on your specific version of Windows, refer to Microsoft support sites.

Other tools

Although VMware recommends the tools listed in this article, other performance collection tools may help you identify issues leading to a fault.

Analyzing the data

When reviewing the collected data:

Look at overall system performance. If the system was running normally when a fault occurred, a hardware issue may be responsible. For more information, see Interpreting an ESX host purple diagnostic screen (1004250).
Look for trends in abnormal behaviour of the system compared to established baselines. When the abnormal subsystem is identified, work to correct the issue.
Look for processes that may be consuming abnormal amounts of RAM or CPU time.
Pay special attention to events that occurred immediately before the fault. Any anomalies may provide clues to the reasons for the fault.

Additional Information

Notes:

For more information, see Interpreting an ESX host purple diagnostic screen
File a Support Request with VMware Support and note this KB article ID in the problem description. For more information, see How to Submit a Support Request
Gather VMware Support Script Data. For more information, see Collecting Diagnostic Information for VMware Products

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box

Collecting performance snapshots using vm-support in ESX and ESXi
使用性能收集工具收集数据以进行故障分析