ESXi hosts get stuck at 19% during entering maintenance mode
search cancel

ESXi hosts get stuck at 19% during entering maintenance mode

book

Article ID: 399791

calendar_today

Updated On:

Products

VMware vSphere ESXi VMware vSphere ESXi 8.0 VMware vSphere ESX 8.x

Issue/Introduction

  • When attempting to place two ESXi hosts within a cluster into maintenance mode, the operation remained stuck at 19%. The vSphere Client displays a message indicating a dependency on a VM that had already been migrated:
    "Waiting for <VM_Name> to complete task: Drm.ExecuteVMotionLRO"

  • Additionally, in the task history, the following message will be observed:
    "The user does not have permission to view the entity associated with this event."

  • Although the VM referenced in the error message had already been evacuated, internal logs showed that the maintenance mode operation neither completed nor failed with an explicit error. The logs also indicated that vCLS was attempting to disable its components on the affected host, which likely blocked the maintenance mode operation.

  • Relevant log excerpts on vCenter, /var/log/vmware/vpxd/vpxd.log
    info vpxd [...] Ignore the WCP enterMaintenanceMode API result for host [...] because vSphere Namespaces is not enabled on its parent cluster.
    info vpxd [...] Started retries to disable vCLS Pod Crx on host [...]
    ..
    ..
    Host has powered-on system VMs; [vim.HostSystem:host-[...],[...]], eam: false, vclscrx: true

Environment

VMware vCenter Server 8.x

Cause

Host remained in an intermediate state while the system attempted to handle vCLS dependencies, which stalled the maintenance mode transition.

Resolution

To resolve the issue:

  1. Log in to the vCenter UI with an Administrator user.
  2. Identify the vCLS VM on the host that is stuck at 19% entering maintenance mode.
  3. Place Cluster vCLS into Retreat Mode
  4. Power off the vCLS VM if they are not automatically cleaned up.
  5. Host will now complete entering Maintenance Mode.
  6. Reboot the ESXi Host.
  7. Exit Maintenance Mode.
  8. Ensure vSAN Skyline Health is 100%
  9. Place Cluster vCLS back into System Managed Mode.

Refer to this doc if the Host is part of a VSAN Node, to place it in Maintenance mode - Place a Member of vSAN Cluster in Maintenance Mode

Additional Information

  • The vCLS VM cannot be manually migrated or powered off via standard methods. It is managed automatically by the vSphere control plane.

  • Post-reboot, maintenance mode operations completed successfully on the affected hosts.

  • Logs did not provide a definitive root cause, but symptoms point toward a vCLS-related blockage.

Powered by Wolken Software