Trivy database download fails with x509 certificate error when using a custom registry - VMware Tanzu Hub

Products

• VMware Tanzu Platform Core

Issue/Introduction

During the deployment or upgrade of the application via Ops Manager, the hubsm-install errand hangs and eventually times out with following error in installation logs.

[i] Waiting for PackageInstall to reconcile...  
           [!] Failed to get PackageInstall sm: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline  
           [x] Installation failed with error: timed out waiting for PackageInstall to reconcile. Last failure: kapp: Error: waiting on reconcile packageinstall/daedalus (packaging.carvel.dev/v1alpha1) namespace: tanzusm:  
             Finished waiting unsuccessfully:  
               Reconcile failed: message: kapp:  
                 Error: waiting on reconcile deployment/daedalus-trivy (apps/v1) namespace: tanzusm:  
             Finished waiting unsuccessfully:  
               Deployment is not progressing:  
                 ProgressDeadlineExceeded, message:  
                   ReplicaSet "daedalus-trivy-####" has timed out progressing.  
         

Checking the Kubernetes pods in the tanzusm namespace reveals the daedalus-trivy-#### pod stuck in an Init:CrashLoopBackOff state. Extracting logs from the download-db init container within that pod shows the following fatal error:

Plaintext

 

FATAL Fatal error run error: init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from ####:####/trivy-db:2: OCI repository error: 2 errors occurred:
* Get "https://####:####/v2/": tls: failed to verify certificate: x509: certificate signed by unknown authority

 

Environment

Tanzu Hub

Cause

This issue occurs because the Allow Insecure Registry Connections setting is disabled in the Tanzu Hub tile configuration, enforcing strict TLS validation.

While the custom CA certificate is present in the BOSH Director's Trusted Certificates configuration, Kubernetes pods run in isolated environments and rely on internal container trust stores. The download-db container does not automatically inherit the host virtual machine's certificates distributed by BOSH. Because there is no native configuration field within the Tanzu Hub tile to inject a custom CA specifically for the Trivy database pod, the strict TLS handshake fails.

Resolution

To allow the container to pull the database from an internal registry using an untrusted or self-signed certificate, bypass the strict TLS verification using the built-in configuration toggle.

1. Navigate to the Ops Manager Installation Dashboard.

2. Select the VMware Tanzu Hub tile.

3. Navigate to Advanced Settings.

4. Locate the Trivy Database Registry Location configuration.

5. Check the box for Allow Insecure Registry Connections.

6. Select Save.

7. Return to the Installation Dashboard.

8. Select Review Pending Changes.

9. Select Apply Changes to restart the deployment and allow the hubsm-install errand to complete successfully.

Additional Information

To verify the exact container causing the errand timeout, run the following kubectl commands from a machine with cluster access:

1. Identify the crashing pod:
   

   kubectl -n tanzusm get pods | grep trivy

2. Extract the logs from the initialization container (replace #### with the exact pod name):
   

   kubectl -n tanzusm logs daedalus-trivy-#### -c download-db