Supervisor upgrade stuck with: Details: Root password sync failed error.

Products

• VMware Tanzu Platform - Kubernetes
• VMware vSphere Kubernetes Service

Issue/Introduction

Supervisor upgrade get stuck with "Details: Root password sync failed" and "Command '/usr/bin/chpasswd' returned non-zero exist status 1.." with below as an example. You may also see the actual Supervisor total upgrade status stuck. It can be in any of the 17 conditions completed stage

 

Environment

vSphere Supervisor 8
vSphere Supervisor 9

Cause

Issue occurs due to Root Disk Space Full at 100% causing to root password sync to fail

Resolution

You can follow below KBs to clear space from Supervisor Control Plane VMs:
https://knowledge.broadcom.com/external/article/383369/vsphere-supervisor-root-disk-space-full.html
https://knowledge.broadcom.com/external/article/381590/supervisor-cluster-unstable-after-upgrad.html

Note that running the first KB may require to reboot the Control Plane VM to free up space as the partition may already be in a read-only mode, preventing actual freeing up of the used blocks without a reboot.