NFS datastore unmount fails with datastore in use, the job is struck at 1% when it is changed to maintenance mode.

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• While moving NFS datastore into Maintenance mode the task gets struck at 1%.
  
  
  
  
• Unmounting datastore fails with error,
  
  
  
  
• Upon checking the open processes can see vCLS VMs in the list:

[root@ESXi host:/var/log] lsof | grep <NFS UUID>
68806641    vmx                   cwd                        -1   /vmfs/volumes/<NFS UUID>/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9
68806641    vmx                   FILE                       43   /vmfs/volumes/<NFS UUID>/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9.vmx.lck
68806641    vmx                   FILE                       47   /vmfs/volumes/<NFS UUID>/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9/vmware.log
68806641    vmx                   FILE                       75   /vmfs/volumes/<NFS UUID>/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9-flat.vmdk
68806641    vmx                   FILE                       76   /vmfs/volumes/<NFS UUID>/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9.nvram
68806641    vmx                   FILE                      108   /vmfs/volumes/<NFS UUID>/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9/vCLS-6852a4cb-8713-4087-95b5-ecf80efdfad9.vmx

Environment

VMware vSphere ESXi

Cause

• If any resources, such as vCLS, files, virtual machines, etc., are still being used within the storage, the NFS Datastore Unmount or State change to Maintenance mode fails.
  
  
• Below Screenshot displays vcls VM still present in the Host,

         

Resolution

1. Make sure the NFS datastore is empty (no read/write data) and free of any active files before moving it to maintenance mode or unmounting it.
2. vCLS VMs are also the cause of the problem. To use retreat mode to turn off vCLS on the cluster refer - Disable vCLS on a Cluster via Retreat Mode
3. Unmount NFS datastore using command line refer - Removing an inaccessible NFS datastore