Unable to remove inaccessible nfs datastores

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms : 

• Unable to remove or unmount nfs inaccessible datastores
• Datastores will show as inaccessible in the vCenter:

• IIn some cases, unmounting a datastore that is in Maintenance Mode may result in the following error.
  
  

Environment

VMware vSphere ESXi (All version)

 

Cause

You cannot unmount an NFS datastore because:

• The NFS datastore has running or registered objects
• The NFS share has been disconnected from the host
• NFS datastore cannot be mounted but is also unavailable in the list to unmount
• This issue can occur if the NFS server is removed without removing the NFS datastore. Re-adding the NFS server does not refresh the datastore mount.
• The datastore also cannot be unmounted if there are any reference stale files present, such as CTK files, snapshots, templates, ISO mappings, or other dependent objects.
  
  

  From the logs, we typically observe a similar event when a reference object is still present on the datastore. 

  vmkernel.log : 

  
  T05:29:58.564Z Wa(180) vmkwarning: cpu82:2103276 opID=a7b3bbf0) WARNING: NFS: 3113: nfs####-####-####-####-###-####-301 has open files, cannot be unmounted. refCount: 3
  
  Hostd.log:
  
  T05:29:58.568Z Wa(164) Hostd[2103276] : [Originator@6876 sub=Hostsvc. DatastoreSystem opID=####-2057626-auto-####-h5: 70230782-c4-b0-b617 sid=525d87f8 user=vpxuser : VSPHERE. LOCAL\Administrator] RemoveDatastore: Failed to remove backend for datastore nfs####-####-####-####-###-####-301 .
T05:30:00.074Z Wa(164) Hostd[2103276]: [Originator@6876 sub=Hostsvc. DatastoreSystem opID=####-2057626-auto-####-h5: 70230782-c4-b0-b617 sid=525d87f8 user=vpxuser : VSPHERE. LOCAL\Administrator] File descriptors in use:/vmfs/volumes/#### ctk.vmdk
T05:30:00.074Z Wa(164) Hostd[2103249] : -- > Alternative report (all opened file handles, full path is not available)

Resolution

Follow the steps in the following KB article to list and remove the inaccessible NFS datastores via an SSH session to an ESXi host that had the datastore mounted on.

Remounting a disconnected NFS datastore from the ESXi command line