Troubleshooting VM Snapshot Consolidation and ESXi Maintenance Failures in VMware Cloud Foundation - VMware vSphere ESXi

Products

• VMware Cloud Foundation
• VMware vSphere ESXi
• VMware vSphere ESXi 8.0

Issue/Introduction

Use this guide to troubleshoot persistent snapshot consolidation and host maintenance mode failures in VMware Cloud Foundation (VCF) environments, where snapshot locks, management agent exhaustion, or stale SDDC inventory may be present.

Environment

VMware Cloud Foundation (VCF) 5.x

VMware vSphere ESXi 7.0, 8.0.

Cause

Multiple layers of failure: orphaned SCSI locks from backup proxies (KB 321365), stale management agent processes (hostd/vpxa) causing vLCM compliance loop errors (KB 320280), or stale host version metadata in SDDC Manager (KB 314168).

Resolution

1. Clear Orphaned Backup Locks:
   1. Identify the backup proxy VM holding the lock via vmkfstools -D.
   2. Unmount the disk from the proxy VM (Do not delete from datastore).
   3. Trigger Snapshot Consolidation in vSphere Client.
2. Restart Management Agents:
   1. If consolidation fails or vLCM errors persist, SSH into the host.
   2. Run /etc/init.d/hostd restart.
   3. Run /etc/init.d/vpxa restart.
3. Sync SDDC Manager Inventory (VCF Environments):
   1. If the host version in SDDC Manager is stale, perform a manual inventory synchronization using the SDDC Manager API.
   2. Verify the host inventory status in the SDDC Manager UI.

Additional Information

SDDC Manager displays incorrect ESXi host version after out of band update

Restarting Management Agents in ESXi

Snapshot consolidation fails due to locks held by third party backup software