Preventing automatic node power-on by the supervisor MachineHealthCheck controller

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

After manually powering off a virtual machine (node) in a Workload Cluster, the Supervisor automatically powers the node back on. This behavior interrupts maintenance windows or intentional shutdowns where a node needs to remain in a powered-off state.

Environment

VMware vSphere Kubernetes Service

Cause

The MachineHealthCheck (MHC) controller monitors the health status of nodes within the cluster. When a node is powered off, it enters a NotReady status. The MHC controller interprets this as a failure and automatically triggers a remediation task (Power On or Recreate) to maintain the cluster's availability and desired state.

Resolution

To persistently keep a virtual machine powered off, update thepowerStatespecification of the VirtualMachine resource. This informs the VM Operator that the desired state isPoweredOff, which prevents the MHC controller from attempting remediation.

1. Log in to a system with kubectl access to the Supervisor Cluster.
2. Identify the VirtualMachine name and the Supervisor namespace where it resides.
3. Run the following command from to patch the resource: kubectl patch virtualmachine <vm-name> --type=merge -p '{"spec":{"powerState":"PoweredOff"}}' -n <NAMESPACE>

Note: The value PoweredOff is case-sensitive and must be capitalized exactly as shown.

To return the node to service, update thepowerStateto PoweredOn with command: kubectl patch virtualmachine <vm-name> --type=merge -p '{"spec":{"powerState":"PoweredOn"}}' -n <NAMESPACE>