HCX - Unable to Power Off Guest OS during Bulk Migration Switchover
Article ID: 321632
Updated On: 03-11-2025
Products
VMware HCX
VMware Cloud on AWS
Issue/Introduction
This article provides information on counter measure for an issue that prevents switchover during bulk migration of HCX.
Bulk Migration fails at the time of cutover with the following error message.
Error message seen in HCX UI:Can not poweroff VM as guest shutdown timed out and forcePowerOff is not set
In the app.log (/common/logs/admin/app.log), a similar error can be found:
2021-03-02 03:12:07.186 UTC [ReplicationTransferService_SvcThread-65273, Ent: HybridityAdmin, , TxId: #####-####-###-823d-94f0a4a331d3] ERROR c.v.h.s.r.j.VirtualmachineOperationJob- Job (#######-####-####-####-175b2cdda7f2) failed with exception Can not poweroff VM as guest shutdown timed out and forcePowerOff is not set java.lang.RuntimeException: Can not poweroff VM as guest shutdown timed out and forcePowerOff is not set at com.vmware.hybridity.service.replicationtransfer.jobs.VirtualmachineOperationJob.doWaitForPowerOffVm(VirtualmachineOperationJob.java:247) at com.vmware.hybridity.service.replicationtransfer.jobs.VirtualmachineOperationJob.run(VirtualmachineOperationJob.java:59) at com.vmware.vchs.hybridity.messaging.LoggingJobWrapper.run(LoggingJobWrapper.java:41Note: Dates, Times and, UUIDs in the log line above are examples.
Environment
VMware HCX
Cause
This issue happens when a guest OS does not reach a fully powered off state within 100 seconds of the initial "Shutdown Guest OS" call as part of bulk migration cutover.
Resolution
Without "Force Power Off" selected, VMware HCX Bulk Migration utilizes VMware Tools to gracefully shutdown a guest OS. As part of this mechanism, HCX will wait 100 seconds for the guest OS to be completely powered down. If the 100 second period elapses and the virtual machine does not reflect a powered off status, the migration will fail. If VMware Tools is not detected on the guest OS, pre-migration validation will prompt a user to select "Force Power Off".
With "Force Power Off" selected, VMware HCX Bulk Migration still utilizes VMware Tools (if present) to gracefully shutdown a guest OS. After 100 seconds has elapsed since the initial "Shutdown Guest OS" call, an immediate power off will be performed (if necessary) on the virtual machine in order to move forward with the migration.
From HCX 4.6.1 release onwards, if the virtual machine does not power off after 100 seconds during the Bulk Migration cut-over stage, HCX waits for an additional 100 seconds to handle any delay in synchronizing with the vCenter Server before failing the migration workflow. Refer to HCX Release Notes.
With "Force Power Off" selected, VMware HCX Bulk Migration still utilizes VMware Tools (if present) to gracefully shutdown a guest OS. After 100 seconds has elapsed since the initial "Shutdown Guest OS" call, an immediate power off will be performed (if necessary) on the virtual machine in order to move forward with the migration.
From HCX 4.6.1 release onwards, if the virtual machine does not power off after 100 seconds during the Bulk Migration cut-over stage, HCX waits for an additional 100 seconds to handle any delay in synchronizing with the vCenter Server before failing the migration workflow. Refer to HCX Release Notes.
Additional Information
Feedback
Yes
No
Powered by
