Virtual Machine unresponsive during snapshot creation/deletion

Products

• VMware vSphere ESXi

Issue/Introduction

• VM Network disruption during the specific period.
• There have been snapshot creation and deletion tasks at the same time.
• The third-party backup software is running the backup task at the same time.
• You can find the familiar log entries at vmware.log during the specific period every day.

  YYYY-MM-DDTHH:MM:SS.517Z In(05) vcpu-0 - CPT: vm was stunned for 40359932 us

Environment

VMware vSphere ESXi 7.0.x

VMware vSphere ESXi 8.0.x

VMware vCenter Server 7.0.x

VMware vCenter Server 8.0.x

Cause

The network disconnect was triggered by the backup task, specifically creating and deleting the snapshot task in the backup progress.

Creating and deleting lots of snapshots will increase the VM's Stun Time. It will suppend VM I/O,make VM in stun status.

Refer to the content in the Additional Information to identify what is virtual machine stun time.

Resolution

From the vSphere side, we recommend some Best Practices to Prevent Prolonged Stunning

Creating Snapshots

• The available storage capacity affects the stun time during snapshot creation.  
• The number of VMDKs attached to the virtual machine impacts the stun time, the fewer, the better.  
• Storage performance influences the stun time.  
• Avoid creating snapshots or backups for multiple virtual machines simultaneously.  

Deleting Snapshots

• Do not create excessive snapshots to prevent performance degradation.  
• Avoid running virtual machines on snapshots for extended periods, as it can lead to increased storage consumption and performance issues.

You can additional details and more clarification at Best practices for using VMware snapshots in the vSphere environment.

Additional Information

What is stun time?

To ensure the consistency of the data when the VM creates a snapshot, it is necessary to ensure that no I/O writes are performed when the snapshot is created, so the VM's I/O will be suspended, putting the VM in a stunned state. 

When a VM is stunned, it may fail to respond as if off the network. This is expected behavior and is temporary.