NetApp NFS41 Server Failover causes NFS41 datastore to go in inaccessible state

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:
Failover and failback operations at array side causes connection loss thereby NFS41 datastore experiences All Paths Down(APD) state.

• VMkernel logs will report following events:

       2018-08-09T06:23:42.864Z cpu2:33728)StorageApdHandler: 1204: APD start for       0x430595afb140 [12345678-561c124e-0000-000000000000]
       2018-08-09T06:23:42.864Z cpu0:33076)StorageApdHandler: 421: APD start event for  0x430595afb140 [12345678-561c124e-0000-000000000000]
       2018-08-09T06:23:42.576Z cpu2:33728)WARNING: NFS41: NFS41FSAPDNotify:5600: Lost  connection to the server 192.168.xx.xx mount point NFS_Datastore, mounted as 12345678-561c124e-0000-000000000000 ("/NFS_Datastore")

• A reboot of ESXi hosts is required to recover from APD.

• This issue does not affect NFSv3

Note :The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5

Cause

After failover NFS41 gets new client ID from NFS server. This causes NFS41 to enter into infinite loop of exchange-id request and reply processing thus causing datastore unavailability.

Resolution

Issue is resolved in ESXi 6.7P02 and ESXi 7.0 GA. The issue has been fixed in upcoming release of ESXi 6.5

Workaround:

Use NFSv3 to prevent the issue.

Additional Information

NetApp KB: VMWare NFSv4.1 Datastores see Disruption During Failover Events Including Upgrade

To troubleshoot NFS APD issues refer to VMware KBs :
Troubleshooting connectivity issues to an NFS datastore on ESX and ESXi hosts
NFS connectivity issues on NetApp NFS filers on ESXi 5.x/6.x
How to troubleshoot NFS APD (All-Paths-Down) issues on VMware ESXi

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice or statements made on third-party websites.Inclusion of such links does not imply that VMware endorses, recommends or accepts any responsibility for the content of such sites.