Latency and intermittent APDs on NFS datastores experienced by ESXi host
search cancel

Latency and intermittent APDs on NFS datastores experienced by ESXi host

book

Article ID: 392787

calendar_today

Updated On:

Products

VMware vSphere ESXi VMware vSphere ESX 8.x VMware vSphere ESX 7.x

Issue/Introduction

  • The VM's may go into a hung state/unresponsive.

  • The VMs running on the ESXi hosts experience slowness and are unable to migrate to any host.

  • Storage vmotion from one NFS share to another takes a long time.

  • Performance has deteriorated messages for the NFS datastores observed in the /var/log/vmkernel.log file of the ESXi host:

            WARNING: NFS: NFSVolumeLatencyUpdate:4879: NFS volume <NFS Volume name> performance has deteriorated. I/O latency increased from average value of 16857(us) to 68722(us). Exceeded threshold 10000(us)

  • Intermittent APD events reported for the NFS datastore in the /var/log/vmkernel.log file of the ESXi host:

            StorageApdHandlerEv: 110: Device or filesystem with identifier [########-########] has entered the All Paths Down state.

  • Running esxtop on the ESXi through CLI reports High DAVG value indicating latency from the storage device (LUN):



Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

 

Cause

  • A temporary network or NFS server availability issue disrupted connectivity between the ESXi host and the NFS server.
  • A faulty port on switch or faulty SFP exists.

Resolution

  1. Reboot all ESXi hosts to clear any residual references for the affected datastores following an APD (All Paths Down).

  2. Engage both the storage and switch vendor to investigate on latency and APD incidents.

Additional Information

Powered by Wolken Software