Supervisor Service Contour installation fails with Image objects stuck in "Resolving" and ImageDisk objects stuck in "Allocating"
Article ID: 443225
Updated On:
Products
Issue/Introduction
When deploying or reinstalling the Supervisor Service Contour, the deployment fails or hangs indefinitely.
Checking the status of the related custom resources reveals that the Image objects are stuck in the Resolving state, while the corresponding ImageDisk objects remain stuck in the Allocating state.
# kubectl get images -A
NAMESPACE NAME STATUS RESOLVED DISK
svc-contour-domain-c100x contour-2b464afcee7d099... Resolving e2e16410ae4996ad1e...
svc-contour-domain-c100x contour-990b67c6f499809... Resolving 905f4d7c9448874fd2...
# kubectl get imagedisks -A
NAMESPACE NAME STATUS DISK
vmware-system-kubeimage e2e16410ae4996ad1e2cd0... Allocating 4d48aa2a-8ac6-43f2-...
vmware-system-kubeimage 905f4d7c9448874fd2c88e... Allocating 0c60558b-f05a-4fe9-...
Environment
vSphere Kubernetes Service
Cause
This issue is caused by a deadlock with stale resources. When the Supervisor Service Contour is uninstalled, its corresponding ImageDisk objects may fail to clean up automatically.
A subsequent reinstallation then attempts to reuse these lingering, hung ImageDisk objects, preventing the new image resolution phase from progressing.
Resolution
To resolve this issue, manually remove the stale ImageDisk objects and refresh the package lifecycle state before attempting a reinstallation.
Step 1: SSH to the Supervisor
ssh root@VCENTER
/usr/lib/vmware-wcp/decryptK8Pwd.py
ssh root@<FIP>
Step 2: Identify the Stale Disks
Run the following command to identify and note the exact names of the Image and RESOLVED DISK objects that are currently stuck.
kubectl get images -n svc-contour-domain-cXXXX
Step 3: Uninstall the Supervisor Service Contour
Uninstall the failed Contour Supervisor Service instance through the vSphere Client UI.
Step 4: Manually Clean Up Stale ImageDisk Objects
Verify if any ImageDisk objects remain in the system. If they are not automatically removed, delete them manually using kubectl:
# List remaining imagedisks
kubectl get imagedisks -n vmware-system-kubeimage
# Delete the stale imagedisk objects found in Step 1
kubectl delete imagedisk -n vmware-system-kubeimage <STALE_IMAGEDISK_NAME>
Step 5: Restart the Spherelet Service
To clear the internal cache on the affected ESXi hosts, log into each ESXi host via SSH and restart the spherelet daemon:
ssh root@ESXI
/etc/init.d/spherelet stop
/etc/init.d/spherelet start
/etc/init.d/spherelet status
Step 6: Reinstall the Supervisor Service Contour
The system will now create fresh Image and ImageDisk objects without encountering conflicts.
Additional Information
Feedback
