ESXi enters Maintenance Mode during Supervisor upgrade in image base cluster

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

During a vSphere Supervisor upgrade, ESXi hosts are observed to automatically enter and exit maintenance mode. This is expected, intended behavior for cluster managed via vSphere Lifecycle Manager (vLCM) image. This article explains why this occurs and how the Supervisor upgrade orchestration interacts with cluster image to ensure host compliance.

Environment

VMware vSphere Kubernetes Service

Resolution

Understanding the Workflow

In vSphere environments using Image-based cluster management, vSphere Lifecycle Manager (vLCM) enforces a strict image compliance model. When initiate a Supervisor cluster upgrade, the orchestration process must ensure that all components including the Kubernetes node agent (Spherelet) and host-level VIBs are successfully updated to match the target cluster version defined by the image.

To perform these updates safely, the host must be in a state where active workloads can be migrated and the underlying software stack can be modified without impacting service availability. Maintenance mode provides this state.

Why This Happens in Image-Based Clusters

The behavior is driven by how vLCM handles host remediation in an Image-based configuration:

• Image Enforcement: Unlike Baseline-based clusters, where manually apply updates, Image-based clusters use an "all-or-nothing" approach to cluster state. The image definition dictates the exact version of the Spherelet and other components that must be present.
• Orchestration Integration: The Supervisor upgrade orchestration works in tandem with vLCM. When the upgrade requires an agent update on the ESXi host, it triggers the vLCM remediation workflow.
• Automatic Remediation: To ensure the host successfully transitions to the target Spherelet version, vLCM automatically places the host into maintenance mode, migrates workloads, applies the update, and exits maintenance mode once the host is compliant with the image.

Comparison: Image-Based vs. Baseline-Based

It is normal for this behavior to differ based on cluster management strategy:

• Image-Based Clusters: Expected behavior. Hosts will automatically enter maintenance mode as part of the orchestration workflow to ensure cluster image compliance.
• Baseline-Based Clusters: Different behavior is observed that ESXi host does not need to enter Maintenance mode, as remediation is often handled differently depending on the specific baseline configurations and manual intervention steps.

Summary

If ESXi hosts enter and exit maintenance mode during a Supervisor upgrade and the upgrade process completes successfully, the system is operating as designed. This process ensures that cluster remains in a supported, compliant state according to the defined cluster image.