Content Library synchronization task failure due to SSL thumbprint mismatch

Products

• VMware vCenter Server

Issue/Introduction

Sync library task fails with the below error:
"A general system error occurred: HTTP request error: cannot authenticate SSL certificate for host content library".

/var/log/vmware/content-library/cls.log

YYYY-MM-DDTHH:MM:SS | DEBUG | tomcat-http-49 | Apache4xRestrictedCipherSSLConnectionSocketFactory | Starting handshake
YYYY-MM-DDTHH:MM:SS | ERROR | tomcat-http-49 | ThumbprintTrustStrategy  | SSL thumbprint mismatch: Received AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA, expected 
BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB
YYYY-MM-DDTHH:MM:SS | ERROR | tomcat-http-49 | VcspClientImpl  | Remote library certificate error: certificate_unknown(46) org.bouncycastle.tls.TlsFatalAlert: certificate_unknown(46)
at org.bouncycastle.jsse.provider.ProvSSLSocketWrap.checkServerTrusted(Unknown Source) ~[bctls-fips-1.0.10.jar:1.0.10]
at org.bouncycastle.jsse.provider.ProvTlsClient$1.notifyServerCertificate(Unknown Source) ~[bctls-fips-1.0.10.jar:1.0.10]
at org.bouncycastle.tls.TlsUtils.processServerCertificate(Unknown Source) ~[bctls-fips-1.0.10.jar:1.0.10]

Environment

VMware vCenter Server 8.x
VMware vSphere ESXI 8.x

Cause

When a Publisher vCenter replaces its Machine SSL certificate, it generates a new unique SHA thumbprint. However, the Subscriber vCenter does not dynamically poll for this change. Because the Subscriber’s database retains the legacy thumbprint associated with the original subscription, the trust relationship is broken, leading to synchronization failures.

• Original Thumbprint: AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA

• Updated Publisher Library Thumbprint: BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB:BB

• Subscriber Library Database Reference: AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA (Mismatch)

To confirm a thumbprint mismatch, perform the following steps:

1. Retrieve the Active Publisher Thumbprint by running the following command from the vCenter SSH to obtain the current SHA1 fingerprint of the Publisher vCenter:
openssl s_client -connect <VC_FQDN>:443 -showcerts </dev/null 2>/dev/null | openssl x509 -noout -fingerprint -sha1

2. Check the Subscriber Database Configuration by logging into the Subscriber vCenter via SSH as root and access the VMware Postgres database (VCDB):

• Access the VCDB:
  /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres


• Query the Content Library table:
  SELECT * FROM cl_library;

3. Compare the thumbprint value in the output against the active thumbprint retrieved in Step 1.

Resolution

The below steps are to make Subscriber Library accept the new certificate.

1. In the vSphere Client, select Content Libraries from the main Menu.
2. Locate the impacted subscribed library, right-click it and choose Edit Settings or select the Actions menu.
3. Click OK without modifying any of the existing settings.
4. A red banner would appear with the following message:
   "SSL certificate cannot be trusted. The thumbprint of the certificate is: [thumbprint]. Do you want to proceed?"
   
   
   
   
5. Click Actions and select Continue to trust the new certificate.
   
   
   
   
6. Click yes to accept the new thumbprint.

Once these steps are completed, the new SSL thumbprint will be updated, and the content library sync task should complete successfully.