Running esxtop in interactive or batch mode fails with out of memory error or Segmentation fault

Products

• VMware vSphere ESXi

Issue/Introduction

To run the performance monitoring utility.

Symptoms:

• Running esxtop in  interactive or  batch mode fails with out of memory error or  Segmentation fault
• In vmkernel.log you will see similar entries as below

 2019-12-16T20:14:56.825Z cpu23:2470253)MemSchedAdmit: 471: Admission failure in path: ssh/esxtop.2470253/uw.2470253
2019-12-16T20:14:56.825Z cpu23:2470253)MemSchedAdmit: 478: UserWorld 'esxtop' with cmdline 'esxtop -a -b -n 1080 -d 2'
2019-12-16T20:14:56.825Z cpu23:2470253)MemSchedAdmit: 489: uw.2470253 (1592799) extraMin/extraFromParent: 1/1, ssh (762) childEmin/eMinLimit: 204800/204800

• In vmkwarning.log you will see similar entries

2019-12-10T18:31:58.513Z cpu32:2322210)WARNING: Heap: 3855: Could not allocate 4096 bytes for dynamic heap cartelheap.2322210. Request returned Admission check failed for memory resource
2019-12-10T18:31:58.513Z cpu32:2322210)WARNING: Heap: 4079: Heap_Align(cartelheap.2322210, 1984/1984 bytes, 8 align) failed. caller: 0x418001f98c95

• This happens when there are NFS datastores mounted on ESXi host.

Note:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Cause

There is a memory leak in the module collecting NFS stats.

Resolution

This issue is resolved in VMware vSphere ESXi 6.7 Patch ESXi670-202008001 .To download go to the Solution Downloads page
This is a known issue in VMware vSphere ESXi 7.0. Currently there is no resolution.

Workaround:
As a workaround run performance monitoring utility using resxtop

•  Launch resxtop and connect to a remote server using command

         resxtop --server <hostname> --username <user>

For example
resxtop -b -a -d 10 -n 1080 -server "IP address" -username root > $(date +%m-%d-%y_%H%M)_resxtop.csv

The above command generates 180 minutes (10 x 1080 seconds) worth of data for analysis.

For more information  refer to Using the resxtop Utility .

Additional Information

Impact/Risks:
None