Host goes into the "Not responding" state when the datastore space utilization reaches near full

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

One or more ESXi host is shown as "Not responding" in vCenter Server though the host(s) are accessible using SSH.

Environment

VMware vSphere ESXi 6.x
VMware vSphere ESXi 7.0
VMware vSphere ESXi 8.0

Cause

The ESXi management agent Hostd reports the host to be unresponsive based on responsiveness of VMFS datastores.

When a VMFS datastore is full or nearly full the hosts connected to that datastore can become unresponsive. This is because the VMFS ondisk resource allocation operations can become very slow - the allocation may need to search the entire datastore space for finding free resources that can be allocated.  The search itself can involve several IOs. This can get be extremely time consuming with a very large datastore. All the hosts sharing a datastore in this state may go unresponsive.

Resolution

This is an expected behavior when the datastore storage space capacity reaches full, while using VMFS5 or VMFS6 datastores.

Workaround:

To work around this issue:

ssh to the esxi host and run below command to check the disk space utilization.
[root@esx-02:~] df -h

1. Monitor and add more capacity as the datastore is getting full.
2. Delete and reduce the unwanted data on datastore to create free space.
3. VMware recommend keeping 10% free space on the datastore.