Virtual Machines Hung Due to Intermittent APDs on NFS Datastores

Products

• VMware vSphere ESXi

Issue/Introduction

• Virtual machines become unresponsive or hang.
• Intermittent APD events reported for the NFS datastore in the /var/log/vmkernel.log file of the ESXi host:
  StorageApdHandlerEv: 106: Device or filesystem with identifier [########-########] has entered the All Paths Down state.
StorageApdHandler: 408: APD start event for 0x4317d5210770 [########-########]

Environment

VMware vSphere ESXi

Cause

A temporary network or NFS server availability issue disrupted connectivity between the ESXi host and the NFS server. During these APD events, the ESXi host cannot communicate with the storage, causing the residing virtual machines to hang while waiting indefinitely for blocked I/O operations to complete.

Resolution

• Reboot the ESXi host to clear any residual references for the affected datastores following an APD (All Paths Down).
• Engage both the storage and switch vendor to investigate on latency and APD incidents.

Additional Information

The ESXi host experiencing latency and intermittent APDs on NFS datastores.