Supervisor warning for secretgen-controller system pod in vSphere Kubernetes Service

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• Following warning message is seen on the supervisor on the vCenter UI
  
  System pod secretgen-controller-#### in namespace vmware-system-appplatform-operator-system is not running.
  
  
• The secretgen-controller pod is in CrashLoopBackOff(CLBO) state.
  
  
• Describing the secretgen-controller pod shows the following warnings:
  
  Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  4m37s                  default-scheduler  Successfully assigned vmware-system-appplatform-operator-system/secretgen-controller-####
  Normal   Pulled     3m54s (x4 over 4m37s)  kubelet            Container image "localhost:5000/vmware/secretgen-controller@sha256:####" already present on machine
  Normal   Created    3m54s (x4 over 4m37s)  kubelet            Created container: secretgen-controller
  Normal   Started    3m54s (x4 over 4m37s)  kubelet            Started container secretgen-controller
  Warning  BackOff    3m28s (x6 over 4m36s)  kubelet            Back-off restarting failed container secretgen-controller in pod secretgen-controller-####_vmware-system-appplatform-operator-system(####)
  Warning  BackOff    2m51s (x3 over 3m17s)  kubelet            Back-off restarting failed container secretgen-controller in pod secretgen-controller-####_vmware-system-appplatform-operator-system(####)
  Normal   Pulled     2m37s (x3 over 3m18s)  kubelet            Container image "localhost:5000/vmware/secretgen-controller@sha256:####" already present on machine

Environment

vSphere Kubernetes Service 

Resolution

Restart the kubelet service on the supervisor and perform a rolling restart of the secretgen-controller deployment.

1. k rollout restart deployment secretgen-controller -n vmware-system-appplatform-operator-system
2. systemctl restart kubelet