High Disk Usage Alert on ESXi OS Partition Due to Stale Coredump File

Products

• VMware vSphere ESXi

Issue/Introduction

A disk usage alert is triggered on an ESXi host OS partition, typically mounted under: /vmfs/volumes/<UUID>.

• When running the following command on the ESXi host:
  
  du -ch /vmfs/volumes/<UUID>
  
  
• You may observe that the vmkdump directory is consuming space in gigabytes, primarily due to a large dump file.
• Command "df -h" shows "OSDATA" partition full or close to being full.
• Navigate "core" directory and find core dump file(s) (e.g., vmkernel-zdump.#) listed:
  
  cd /var/core/

ls -lah

 

Log errors seen:

• /var/run/vobd.log
  
  [vob.user.scratch.partition.full] Scratch is low on available space (## MiB free).This may result in system failure. Please refer to KB article: KB 87212

Environment

VMware vSphere ESXi 7.x 

VMware vSphere ESXi 8.x 

Cause

• The issue is caused by a stale core dump file that was generated during a previous system event or crash and was not automatically removed. This file continues to consume space on the OS partition, leading to disk usage alerts.
• Steps were followed from KB Generating a VMkernel zdump manually from a dump file in ESXi host to create a "vmkernel-zdump" file from the live core file and this file was not manually removed once downloaded for review.

Resolution

To resolve the high disk usage and clean up the stale live coredump file:

1. Unconfigure the current coredump file setting:
   
   esxcli system coredump file set --unconfigure
   
   

2. Delete the stale dump file:
   
   rm -rf /vmfs/volumes/<datastore-name>/<dumpfile-name>
   
   

3. Reconfigure the coredump file (optional but recommended):
   
   esxcli system coredump file set --configure

esxcli system coredump file set -p /vmfs/volumes/<datastore-name>/<new-dumpfile-name>
   
   

4. Verify configuration:
   
   esxcli system coredump file list

 

To delete the "vmkernel-zdump.#" file that is no longer needed:

1. Navigate to the core dump file location:
   
   cd /var/core/
   
   
2. List the files in the directory:
   
   ls -lah


3. Remove the core dump file:
   
   rm <vmkernel-zdump_file_name>

Additional Information

• Deactivate and Delete a Core Dump File – Broadcom TechDocs
• Configuring ESXi coredump to file instead of partition