ESXi host status "Not Ready" in Supervisor Cluster

Products

• Tanzu Kubernetes Runtime
• VMware vSphere Kubernetes Service

Issue/Introduction

• In a VMware Cloud Foundation (VCF) or vSphere with Tanzu environment, an ESXi host may transition to a Not Ready state. This prevents the Supervisor Cluster from scheduling workloads (vSphere Pods, Tanzu Kubernetes Grid clusters) on the affected host.
• The issue can be verified by running the below command in the supervisor cluster.

root@<supervisor-id> [ ~ ]# kubectl get nodes

NAME                STATUS           ROLES                 AGE     VERSION

<supervisor-1>   Ready      Control-Plane,master   <days>   <supervisor-version>
<supervisor-2>   Ready      Control-Plane,master   <days>   <supervisor-version>
<supervisor-3>   Ready      Control-Plane,master   <days>   <supervisor-version>
<esxi-host-1>    Ready      agent                  <days>   <spherelet-version>
<esxi-host-2>    Ready      agent                  <days>   <spherelet-version>
<esxi-host-3>    NotReady   agent                  <days>   <spherelet-version>

• The below command can be run in the affected ESXi host.

root@<esxi-host># /etc/init.d/spherelet status

YYYY-MM-DD hh:mm:ss init.d/spherelet spherelet is not running

Environment

• vSphere 8.x
• vCenter 8.x

Cause

The Spherelet service which acts as the kubelet equivalent for ESXi is not running. This prevents the host from sending health heartbeats to the Supervisor control plane, leading to the "Not Ready" state.

Resolution

Follow these steps to start the service and restore host connectivity in the Supervisor:

1. Log in to the affected ESXi host as root via SSH.
2. Check if the Spherelet service is active by running the below command:
   /etc/init.d/spherelet status
   
   
3. If the service is stopped, start it manually:
   /etc/init.d/spherelet start


4. Wait 30 seconds and verify the service remains in a "running" state:
   /etc/init.d/spherelet status
   
   
5. Return to the vSphere Client UI or run the kubectl get nodes command in the supervisor to confirm the host transitions back to Ready state. This typically occurs within 1–2 minutes of service restoration.

Additional Information

Japanese KB: スーパーバイザクラスタに参加しているESXiホストが"Not Ready"状態になる