Using performance collection tools to gather data for fault analysis
search cancel

Using performance collection tools to gather data for fault analysis

book

Article ID: 308926

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

This article explains how to use performance collection tools to gather data for analysis of faults such as:
  • Unresponsive ESX hosts
  • Unresponsive virtual machines
  • ESX host purple diagnostic screens


Symptoms:



Environment

VMware ESXi 3.5.x Embedded
VMware ESX 4.1.x
VMware ESXi 3.5.x Installable
VMware vSphere ESXi 5.0
VMware ESXi 4.0.x Embedded
VMware ESXi 4.1.x Embedded
VMware ESX Server 3.5.x
VMware ESX Server 3.0.x
VMware ESX Server 2.5.x
VMware vSphere ESXi 5.1
VMware ESXi 4.1.x Installable
VMware ESX 4.0.x
VMware ESXi 4.0.x Installable

Resolution

Why gather performance data for a fault?

If the diagnostic logs do not help you determine the cause of a fault, you may need to use performance collection tools to gather further data for analysis.
Set up performance collections tools to gather data about faults that may occur.

Performance gathering tools

VMware recommends the following tools for gathering performance data:

top

The top utility provides a list of CPU-intensive tasks for the ESX host Service Console.
Use top in batch mode for Fault troubleshooting by directing the output to a file so that it can be reviewed after a recurrence.
Note: The top command is not available for ESXi.
To run the top utility, run the command:
# top –bc –d <delay in seconds> [–n <iterations>] > output-perf-stats-file.txt
Use the information in the output file to identify any trends before the fault.

esxtop

The esxtop tool provides performance statistics for the entire ESX/ESXi host. It provides details of network, storage, CPU, and memory load from the VMkernel perspective. It provides details on a VMkernel world basis.
To collect the data over long periods of time, run esxtop in batch mode. Direct the output to a file so that it can be reviewed after the fault.
To run the esxtop tool, run the command:

# esxtop –b –d <delay in seconds> [-n <iterations>] > output-perf-statistics-file.csv

Note:
  • For ESX/ESXi 3.0.0 and earlier versions, the output is in plain text. For ESX/ESXi 3.0.0 and later versions, the output is in Comma-Separated Values (CSV). This file can be read in Microsoft's Performance Monitor or in a spreadsheet utility.
  • Make sure you have enough free space available on the location from where you are running the command and on the datastore where the output file saved. Also make sure there is no performance issue on the datastore where the output file is saved.

resxtop

Like esxtop, the resxtop tool provides performance statistics for a specified ESX host in the network. It provides the same performance information as esxtop and may be used either after deploying the VMware vSphere Management Assistant (vMA) virtual appliance or installing the VMware Command-Line Interface (vCLI).

resxtop is particularly helpful for ESX and ESXi deployments where an interactive shell login is unavailable.

To run the resxtop tool and collect batch performance data, log into the vMA or open the vCLI, and execute the command:

# resxtop [server] [vihost] [portnumber] [username] -b -d <delay in seconds> [-n <interations>] > output-perf-statistics-file.csv

Note: For additional information on using resxtop, see the Resource Management Guide.

vm-support -s

Use the vm-support command with the -s parameter to collect performance statistics, system configuration information, and logging. Submit the file generated by this command to VMware Support for further assistance, if required.

For information on the vm-support command in ESXi 5.0, see Collecting performance snapshots using vm-support (1967).

Performance Monitor (PERFMON.EXE)

Microsoft's Performance Monitor is a utility that comes with every Microsoft Windows NT-based Operating System. This utility can be used to monitor local and remote Microsoft Windows machines. It can log performance data and display data from logs or real-time data.
This utility is useful when reviewing data collected from the esxtop tool and for troubleshooting virtual machine unresponsiveness. When using Performance Monitor for virtual machine unresponsiveness, collect the data remotely from another Microsoft Windows machine so that the utility does not affect the data being gathered.
For more information about Performance Monitor on your specific version of Windows, refer to Microsoft support sites.

Other tools

Although VMware recommends the tools listed in this article, other performance collection tools may help you identify issues leading to a fault.

Analyzing the data

When reviewing the collected data:
  • Look at overall system performance. If the system was running normally when a fault occurred, a hardware issue may be responsible. For more information, see Interpreting an ESX host purple diagnostic screen (1004250).
  • Look for trends in abnormal behaviour of the system compared to established baselines. When the abnormal subsystem is identified, work to correct the issue.
  • Look for processes that may be consuming abnormal amounts of RAM or CPU time.
  • Pay special attention to events that occurred immediately prior to the fault. Any anomalies may provide clues to the reasons of the fault.


Additional Information

Notes:

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box



Recording ESX or ESXi VMkernel state with vsi_traverse
パフォーマンス収集ツールを使用した障害分析データの収集
Collecting performance snapshots using vm-support in ESX and ESXi
使用性能收集工具收集数据以进行故障分析

Powered by Wolken Software