Virtual Machine fails to power on with "Failed to initialize swap file" or "monitorloop power on failed" within the UI alongside with "No free handles" within /var/log/vmkernel.log

Products

• VMware vSphere ESXi

Issue/Introduction

When attempting to power on a Virtual Machine residing on a vSphere Virtual Volumes (vVols) datastore, the operation fails.

The vCenter Server task pane reports the following error: Failed to initialize swap file '/vmfs/volumes/vvol:<DATASTORE_UUID>/<VM_NAME>.vswp' : Failure

Reviewing the ESXi host logs reveals the following:

• /var/log/vvold.log confirms the storage array successfully created the swap object (FINAL SUCCESS createVirtualVolume), but the host fails to bind it.

• /var/log/vmkernel.log reports a handle exhaustion error during device creation: Creation of device <UUID> with device lib failed: No free handles Error returned from vmklink call: Failure

• Executing either of the below commands:
  
  For vvol:
  ls -l /vmfs/devices/vvol/ | wc -l

For non-vvol:
ls -l /vmfs/devices/ | wc -l 

  via SSH on the affected host returns a value of more than 4096 or 4097.

Environment

VMware vSphere ESXi  8.x, 9.x
VMware vSphere Virtual Volumes (vVols)
ESXi hosts with very high number of attached datastores

Cause

This issue occurs when the ESXi host exhausts its internal allocation of storage device tracking handles.

In a vSphere Virtual Volumes architecture, every virtual disk (VMDK), active snapshot delta disk, and swap file (.vswp) registers as a uniquely bound vVol (sub-LUN). The ESXi host device library (disklibk) enforces a strict architectural limit for bound devices/paths (e.g., 4096 objects). When an ESXi host reaches this configuration maximum, the kernel rejects new binding requests with the VMK_NO_FREE_HANDLES error. Consequently, the swap file cannot be initialized, and the Virtual Machine power-on operation fails.

Resolution

To resolve the issue, you must reduce the number of active vVol bindings on the affected ESXi host.

Check the host(s) that have higher number of disks attached with the command, where below anything higher than 4096 will confirm that the limit is reached. For example:

For vvol:
ls -l /vmfs/devices/vvol/ | wc -l

For non-vvol:
ls -l /vmfs/devices/ | wc -l 


Immediate Workarounds:

1. Free up Host Handles (vMotion): Perform a live vMotion of several existing, powered-on Virtual Machines from the affected ESXi host to other hosts in the cluster. This unbinds their associated vVols from the source host, immediately freeing up device handles for new operations.

2. Put the host into maintenance: For the host having breached the limit, and vMotions are not sufficient to free handles, they may have been locked with zombie processes/previous tasks. Put the host into maintenance mode and move VMs away, then reboot the host to clear the handles.

Long-Term Resolution: Rebalance the Virtual Machine distribution across the cluster to ensure no single ESXi host continuously operates at the configuration maximum for bound vVols. Consolidate active Virtual Machine snapshots, as each snapshot delta disk consumes an additional device handle while the Virtual Machine is powered on.

You can connect to each host in the environment and then run the above command to identify the number of objects that exist for each host.

Additional Information

DRS and Placement Recommendations: vSphere Distributed Resource Scheduler (DRS) natively calculates workload placement based on CPU and Memory utilization. It does not actively track or balance Virtual Machines based on underlying vVol/sub-LUN counts. To prevent this issue from recurring in vVol environments, implement the following strategies:

• VM Anti-Affinity Rules: Configure DRS Anti-Affinity rules for Virtual Machines that consume a disproportionate number of vVols (e.g., database servers with many VMDKs or Virtual Machines requiring deep snapshot chains). This forces them onto separate ESXi hosts, physically distributing the storage device handle load.

• Host Groups / Affinity Rules: Divide heavy vVol consumers across logical ESXi host groups using "Should run on hosts in group" rules. This guides DRS to distribute the storage handle load evenly without strictly violating High Availability (HA) constraints.

• Stagger Backups: Simultaneous backup jobs can trigger massive, sudden spikes in vVol handle consumption due to temporary snapshot creation. Stagger backup schedules across different times and hosts to prevent exceeding the host limits.

• Configuration Maximums: Refer to the VMware Configuration Maximums portal for exact limits on "Maximum bound vVols per host" for your specific ESXi version.