Expired signing certificate for all published VKRs. VKS clusters cannot be created or updated.
search cancel

Expired signing certificate for all published VKRs. VKS clusters cannot be created or updated.

book

Article ID: 424472

calendar_today

Updated On:

Products

VMware vSphere Kubernetes Service

Issue/Introduction

The signing key used for every published VKR expired on January 3, 2026. These can no longer be synced to or uploaded to content libraries that are configured to enforce certificate validity, i.e., have a Security Policy enabled. Content libraries that do not have a security policy enforced will not be affected.

In /var/log/vmware/content-library/cls.log on the vCenter, we see similar errors.

[timestamp]  | DEBUG    | null             | type-adapter-2            | CertUtils                      | primaryCert certificate has expired on 1/3/26 10:17 PM
[timestamp] | DEBUG    | null             | type-adapter-2            | CertUtils                      | primaryCert certificate is expired: -----BEGIN CERTIFICATE-----
---------------------------------------------------
-----END CERTIFICATE-----
[timestamp] | ERROR    | null             | type-adapter-2            | ManifestCertServiceImpl        | Certificate photon-ova.cert validation failed. Error: [CERTIFICATE_EXPIRED]
[timestamp] | WARN     | null             | type-adapter-2            | CertValidateResult             | CertValidateResult - getResult called - certErrs existed

Cause

This issue is caused by the expiration of the signing key

Resolution

Broadcom Engineering team is aware of this issue and is working on a permanent fix involving reprovisioning signing keys that are valid. 

Implement one of the following workarounds:  

Workaround 1 (Preferred): 

Edit the existing Content Library and uncheck "Apply Security Policy"

  1. Navigate to the Content Library which is configured for the Supervisor/VKS Cluster (identified per screenshot below).
  2. Click on Actions.
  3. Edit Settings 
  4. Uncheck "Apply Security Policy".  
  5. Click OK.

Workaround 2:

  1. Edit the content library, set the content library settings to disable “Apply Security Policy”. Click OK. Ensure that the “Update Library” task completes successfully.
  2. Edit the content library settings again to enable “Apply Security Policy”. Click OK. Ensure that the “Update Library” task completes successfully.
  3. Verify from the OVF templates page of the content library that the images are security compliant now.
  4. With Supervisor Kubernetes context, run below commands to update the cclitem objects:
    1. Get CL ID(s) of the content libraries subscribed to CDN:

      kubectl get contentlibraries.imageregistry.vmware.com -A -o json | jq -r '.items[] | select(.status.subscriptionInfo.URL == "https://wp-content.vmware.com/v2/latest/lib.json" or .status.subscriptionInfo.URL == "https://wp-content.broadcom.com/v2/latest/lib.json") | .metadata.name'

    2. For each content library found in above command, run below command: 

      kubectl get cclitem -o json | jq -r '.items[] | select(.status.contentLibraryRef.name == "example-CL-ID") | .metadata.name' | xargs kubectl delete cclitem

      Note: Replace example-CL-ID with the output from the previous command.

    3. Delete the imageregistry pods on the Supervisor namespace with the below command: 

      kubectl get pods -n vmware-system-imageregistry --no-headers | awk '{print $1}' | xargs kubectl delete pod -n vmware-system-imageregistry