Error: Quick stats not up-to-date alarm during VM consolidation in ESXi

Products

• VMware vSphere ESXi
• VMware Telco Cloud Platform

Issue/Introduction

Symptoms:

1. Backup-side snapshots fail to delete, causing virtual machines to become unresponsive and host management agents to hang during restart due to task backlog.
   
   
2. Snapshot consolidation fails or remains in a pending state.
   
   
3. The ESXi host displays the warning: Quick stats on <hostname> is not up-to-date.
   
   
4. Host management agents (hostd or vpxa) may become unresponsive or cause the host to show as "Not Responding" in vCenter.
   
   
5. hostd.log contains entries similar to:
   
   

   ####-##-##T##:##:##.###Z Wa(###) Hostd[########]: [Originator@#### sub=IoTracker] In thread ########, stat("/vmfs/volumes/####-####/####-####_1-000001-sesparse.vmdk") took over 306972 sec.

Environment

VMware ESXi: 7.0 U2 and higher.

Telco cloud platform: 5.0

Cause

Environments utilizing SIOC policies with limits/shares below 400 may experience consolidation tasks that overwhelm the host management agents, leading to a hung state or stale statistics.

Resolution

Fixed in release ESXi 7.0 U3i and 8.0b and higher. See KB# 142814 to download this release

Workaround:

1. Reboot the impacted ESXi host to restore statistics collection.
   
   
2. Disable SIOC policies or increase the Shares and Limits above 400 that will allow consolidation to work.
   
   

Additionally, cloning the affected VM successfully resolves the behavior. Furthermore, the issue does not manifest when SPM is disabled, when utilizing policies with higher resource limits (such as Gold), or when the lead policy is reapplied.

Additional Information

Impact/Risks:

Following an upgrade to ESXi 7.0U2 or later, environments utilizing SIOC policies with limits or shares below 400 may experience host hangs during VM consolidation tasks.