Virtual machines residing on NFS storage become unresponsive during a snapshot removal operation

Products

• VMware vSphere ESX 6.x
• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x

Issue/Introduction

When using NFSv3 to mount NFS datastores, and a VADP-enabled backup solution that uses the hot-add method, you experience these symptoms:

• During a snapshot creation or removal operation, virtual machines residing on NFS storage become unresponsive, it is also observed ESXi host also going unresponsive at the same time.
• When removing snapshots after backing up a virtual machine residing on an NFS datastore using a backup application, the virtual machine becomes unresponsive for approximately 30 seconds.
• Removing snapshots after backing up a virtual machine residing on an NFS datastore takes a long time to complete.
• This issue occurs when the target virtual machine disk was hot-added for the backup/restore operation.

Cause

This issue occurs when the target virtual machine and the backup appliance reside on two different hosts, and the NFSv3 protocol is used to mount NFS datastores. A limitation in the NFSv3 locking method causes a lock timeout, which pauses the virtual machine being backed up.

Resolution

This issue is resolved in VMware ESXi 8.0 Update 2b release. To download go to Download Broadcom products and software

If the issue is observed on VMware ESXi 8.0 Update 2b or a later version, please refer to KB 428200.

To work around this issue, use one of the below workarounds.

• Ensure that the source virtual machine resides in the same ESXi host where the backup appliance is running.
• Consider using the NFSv4 protocol for mounting NFS volumes as NFS 4.1 uses native protocol specified locking - Refer to NFS File Locking - Guidelines and Requirements for NFS Storage with ESXi
  Note: Before implementing refer to NFS Protocols and vSphere Solutions - NFS Datastore Concepts and Operations in vSphere Environment
  
  
• Use LAN/NBD transport that uses the NFC (Network File Copy) in your backup solution or disable SCSI hot-add through the backup software. For more information about NBD transport, see the Virtual Disk Transport Methods section in the Virtual Disk Development Kit (VDDK) Programming Guide
  
  

Note: NFSv4 is released with ESXi 6.0, and later. It is not available in earlier versions.
          The issue might still occur in rare cases for workaround "Use LAN/NBD transport that uses the NFC (Network File Copy) in your backup solution or disable SCSI hot-add through the backup software".