Performing cold migration to other datastores or cloning of a virtual machine fails when using NetApp NFS storage in cluster-mode with VAAI enabled

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Performing cold migration to other datastores or cloning of a virtual machine fails in ESXi host 5.5 Update 1 when using NetApp FAS6220 array with ONTAP 8.2.1 NFS storage in cluster-mode with VAAI enabled
• In vSphere, you see this error:
  
  Error caused by file /vmfs/volumes/11ddde9e-3a17048b/<VM>/<VMDKNAME>.vmdk
   
• The virtual machine is in power off state and there is no locking file in the folder of the virtual machine. Disk files (.vmdk) are in Normal state.
• Performing Storage vMotion (when the virtual machine is in power on state) and copying files over datastore browser to another datastore works.
• This issue does not occur in NetApp NFS in 7-mode.
• Hardware acceleration is in supported status on the NFS datastore.
• In the hostd log file (in the ESXi host), located at /var/log/, you can see entries similar to:
  
  <YYYY-MM-DD><time> [4EEC2B70 error 'Vmsvc.vm:/vmfs/volumes/11ddde9e-3a17048b/<VM>/<VM>.vmx' opID=hostd-b56b user=vpxuser] Exception Unknown error trying to get sizes from /vmfs/volumes/11ddde9e-3a17048b/<VM>/<VMDKNAME>.vmdk
  <YYYY-MM-DD><time> [4EEC2B70 error 'Vmsvc.vm:/vmfs/volumes/11ddde9e-3a17048b/<VM>/<VM>.vmx' opID=hostd-b56b user=vpxuser] Exception Unknown error trying to get sizes from /vmfs/volumes/11ddde9e-3a17048b/<VM>/<VMDKNAME_1>.vmdk
  <YYYY-MM-DD><time> [4EEC2B70 info 'Hostsvc' opID=hostd-b56b user=vpxuser] Lookupvm: World ID not set for VM 1
  <YYYY-MM-DD><time> [4EEC2B70 verbose 'Hbrsvc' opID=hostd-b56b user=vpxuser] Replicator: VmFileProviderCallback VM (id=1)
  
  </time></time></time></time></time></time>
• In the vmkernel log file, located at /var/log/, you can see entries (VAAI related issues) similar to:
  
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: vendor_nas_plugin: SUCCESSES: RsrvSpace [0] Cln-Full
  [0] Cln-Lazy [0] cln-DRun [0], Ext-stats [2081]
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: vendor_nas_plugin: FAILURES: RsrvSpace [0] Cln-Full [
  0] Cln-Lazy [0] cln-DRun [0], Ext-stats [43577]
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: SvaNasPlugin: SUCCESSES: RsrvSpace [0] Cln-Full [0] C
  ln-Lazy [0] cln-DRun [0], Ext-stats [0]
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: SvaNasPlugin: FAILURES: RsrvSpace [0] Cln-Full [0] Cl
  n-Lazy [0] cln-DRun [0], Ext-stats [0]
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: vmfsNasPlugin: SUCCESSES: RsrvSpace [0] Cln-Full [0]
  Cln-Lazy [0] cln-DRun [0], Ext-stats [0]
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: vmfsNasPlugin: FAILURES: RsrvSpace [0] Cln-Full [0] C
  ln-Lazy [0] cln-DRun [0], Ext-stats [0]
  <YYYY-MM-DD><time> [4EEC2B70 info 'Libs' opID=hostd-b56b user=vpxuser] VAAI-NAS :: NAS Mapping Used successfully for 47153 times</time></time></time></time></time></time></time>

Environment

VMware vSphere ESXi 5.5
VMware vSphere ESXi 6.0

Resolution

Note: This is not a VMware software issue. This solution is not directly supported by VMware. For issues with this configuration, contact NetApp directly to change the configuration in the array.

 

To work around this issue, change the configuration in the array as NFSv3 and v4 are required for proper functioning of NFS VAAI.

Additionally, to allow VAAI calls, the NFSv4 protocol needs to be allowed.

 

To allow NFSv4 protocol, use one of these options:

• Set the allowed protocol in the rule to NFS (that encompasses all versions of NFS).
• Set it to NFSv3 or NFSv4 to include both version 3 (for the datastore access) and NFSv4 for the VAAI specific calls.

 

Note: The preceding link was correct as of October 20, 2014. If you find the link is broken, provide feedback and a VMware employee will update the link.

Note: Ensure to enable NFSv4 in the NetApp array only, while ESXi hosts are still using NFSv3 (Currently, NetApp only supports NFSv3 for ESXi 5.5 Update 1).

Additional Information

クラスタモードの NetApp NFS ストレージで VAAI が有効になっていると、仮想マシンの他のデータストアへのコールドマイグレーションやクローンに失敗する