I/O operation from the ESXi host to the datastore failed with "Input/Output" error.

Products

• VMware

Issue/Introduction

Symptoms:

• Error: "There is no more space for virtual disk Disk_Name.vmdk. You might be able to continue this session by freeing disk space on the relevant volume, and clicking Retry. Click Cancel to terminate this session." are noticed on the VM's despite having ample free space on the datastore.
  
  
• Creation of directory on the concerned datastore along with any I/O operation failed with "Input/Output" error.

Environment

VMware ESXi 7.x

VMware ESXi 8.x

VMware ESX 9.x

Cause

The cause for this issue is the underlying Data Pool on the storage array is full. In modern storage systems, "Thin Provisioning" allows the user to "Subscribe" or over-allocate more space to the ESXi hosts than the storage array actually have physically available.

When the physical pool completely filled up, the storage array cannot write any new data blocks and locked the LUNs to "Read-Only" to protect data integrity. Because it cannot complete a "Write" operation, it sends an I/O Error back to the ESXi host.

Resolution

1. Delete Unnecessary Snapshots

• The fastest way to recover space is to delete old snapshots on the storage array, deleting them will immediately return physical blocks to the "Free" pool.

2. Check for "Orphaned" LUNs

• If any of the LUN's are no longer in use by ESXi, delete them from the array to free up the space they are reserving.

3. Add Physical Disks

• If the data currently stored is all necessary, the only permanent hardware fix is to add more physical drives to this pool to increase the physical storage space.