ESXi 9.1 remediation fails due to DRS violations or "Hot migration is disabled"

Products

• VCF Operations

Issue/Introduction

In VMware Cloud Foundation (VCF) 9.1, ESXi host remediation to version 9.1.0.0.200 may fail.

This occurs when hosts cannot enter maintenance mode due to DRS rule violations, or when subsequent vMotion operations are blocked because the migration module is disabled.

Remediation fails with the following error in the VCF UI or logs:

Failed getting host recommendation from DRS to enter maintenance mode... Reason: 'This operation would violate a virtual machine affinity/anti-affinity rule.'

Hosts are skipped after multiple retry attempts: Skipping remediating the following host(s) #### since no more retry attempts are available.

Manual vMotion fails with the error: Host 'Host FQDN' does not support this operation because it is disabled in vmkernel.

Environment

VMware Cloud Foundation 9.1

VMware vSphere ESXi 9.1.x

Cause

 Virtual machine affinity or anti-affinity rules prevent DRS from automatically evacuating VMs to other hosts, blocking the host from entering maintenance mode.

A previously failed remediation attempt can leave the VMkernel migrate module in a disabled state (Enabled = 0), preventing the necessary vMotion operations for subsequent upgrade retries.

Resolution

To resolve this issue and complete the upgrade, follow these steps:

Address DRS Rule Violations

1. Log in to the vSphere Client.
2. Navigate to the affected Cluster > Configure > Configuration > VM/Host Rules.
3. Temporarily disable any Affinity or Anti-Affinity rules that restrict VM movement.
4. Ensure there are enough available hosts in the cluster to satisfy the resource requirements of the running VMs.

Re-enable the Migration Module

1. Connect to the host via SSH as root.

2. Run the following command to check the migration status: vsish -e get /config/Migrate/intOpts/Enabled
   Enable the module, If the current value returned is 0, run this command to enable it.
   vsish -e set /config/Migrate/intOpts/Enabled 1

3. Ensure the module loads on boot by running esxcli system module set -e true -m migrate

4. Restart the hostd & vpxa services of the host.
   /etc/init.d/hostd restart
/etc/init.d/ vpxa restart

5. Retry the virtual machine migration to the host.

Retry Upgrade Once rules are adjusted and the migration module is enabled, retry the remediation from the SDDC Manager or VCF Lifecycle UI.

Note: After a successful upgrade, remember to re-enable any VM/Host rules that were disabled.