Tanzu Hub Tile Hubsm-Install Errand Fails - failed to download vulnerability DB

Products

VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

When installing the Tanzu Hub Tile, the deployment may fail when running the errand to install and configure packages.

Deployment Change Log

===== 2025-07-17 23:33:18 UTC Running "/usr/local/bin/bosh --no-color --non-interactive --tty --environment=##.###.##.### --deployment=hub-9f57347adb065739e14b run-errand hubsm-install --instance registry/first"
Using environment '##.###.##.###' as client 'ops_manager'

Using deployment 'hub-9f57347adb065739e14b'

Task 669

Task 669 | 23:33:18 | Preparing deployment: Preparing deployment (00:00:02)
Task 669 | 23:33:20 | Running errand: registry/########-####-####-####-######## (0) (02:02:20)
Task 669 | 01:35:40 | Fetching logs for registry/########-####-####-####-######## (0): Finding and packing log files (00:00:01)

Task 669 Started  Thu Jul 17 23:33:18 UTC 2025
Task 669 Finished Fri Jul 18 01:35:41 UTC 2025
Task 669 Duration 02:02:23
Task 669 done
...
...
Stderr     kctrl: Error: packageinstall/sm (packaging.carvel.dev/v1alpha1) namespace: tanzusm:  
            Reconciling: kapp:  
              Error: waiting on reconcile packageinstall/daedalus (packaging.carvel.dev/v1alpha1) namespace: tanzusm:  
            Finished waiting unsuccessfully:  
              Reconcile failed: message: kapp:  
                Error: waiting on reconcile deployment/daedalus-trivy (apps/v1) namespace: tanzusm:  
            Finished waiting unsuccessfully:  
              Deployment is not progressing:  
                ProgressDeadlineExceeded, message:  
                  ReplicaSet "daedalus-trivy-cbfdfbff4" has timed out progressing.. Reconcile failed: Error (see .status.usefulErrorMessage for details)

In the full error output, the replicaset that is timing out is related to 'daedalus-trivy-cbfdfbff4'.

To find out more information on this replicaset/deployment, ssh into the registry VM:

SSH into the registry VM (registry/########-####-####-####-########)

```
sudo su
```

export KUBECONFIG=//var/vcap/jobs/hubsm-install/config/kubeconfig

alias k=/var/vcap/packages/kubernetes/bin/kubectl

k -n tanzusm describe deployment daedalus-trivy

```
k get events -n tanzusm
```

k get events -n tanzusm

LAST SEEN   TYPE      REASON                            OBJECT                                   MESSAGE

5m48s       Normal    Pulled                            pod/daedalus-b689c7b8d-r4jl7             Container image "registry.internal:10500/hub-self-managed/repo@sha256:12c396bd585df7ec21d5679bb6a83d4878bc4415ce926c9e5ea6426d23c60bdc" already present on machine

5m48s       Normal    Created                           pod/daedalus-b689c7b8d-r4jl7             Created container: wait-for-endpoints

4m20s       Warning   BackOff                           pod/daedalus-b689c7b8d-r4jl7             Back-off restarting failed container wait-for-endpoints in pod daedalus-b689c7b8d-r4jl7_tanzusm(0504faa2-b90a-4034-877f-b6881844bde1)

10m         Normal    Pulled                            pod/daedalus-trivy-cbfdfbff4-l6lph       Container image "registry.internal:10500/hub-self-managed/repo@sha256:deb04ad5432594979305c6ee0153e915ed8fb64052e640a8fa0eec2789ee19cb" already present on machine

20m         Normal    Created                           pod/daedalus-trivy-cbfdfbff4-l6lph       Created container: download-db

20m         Normal    Started                           pod/daedalus-trivy-cbfdfbff4-l6lph       Started container download-db

10m         Warning   BackOff                           pod/daedalus-trivy-cbfdfbff4-l6lph       Back-off restarting failed container download-db in pod daedalus-trivy-cbfdfbff4-l6lph_tanzusm(ea8c980a-240d-417d-b701-373f2c141399)

2m24s       Warning   Unhealthy                         pod/ensemble-ui-59bfb45cc6-dhlf5         (combined from similar events): Liveness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "5b43266ce9dfbca8d442e1212725c04e45ac47c2f7ea461c298434ec506c4f69": OCI runtime exec failed: exec failed: unable to start container process: exec: "cat": executable file not found in $PATH: unknown

2m28s       Warning   FailedToRetrieveImagePullSecret   pod/postgres-operator-79597b57b4-qj246   Unable to retrieve some image pull secrets (regsecret); attempting to pull the image may not succeed.

From the event output, we can see two containers that fail and Back-Off.

Back-off restarting failed container download-db in pod daedalus-trivy-cbfdfbff4-l6lph

Back-off restarting failed container wait-for-endpoints in pod daedalus-b689c7b8d-r4jl7_tanzusm

Drill down further into the container to see why it failed.

kubectl -n tanzusm logs daedalus-trivy-cbfdfbff4-l6lph -c download-db

In the following error, the container download-db is trying to pull from Google Container Registry (ghcr.io) but timing out.

2025-07-21T16:23:31Z    FATAL   Fatal error init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from ghcr.io/aquasecurity/trivy-db:2: OCI repository error: 1 error occurred:

    * Get "https://ghcr.io/v2/": dial tcp ###.##.###.##:443: connect: connection timed out

Environment

Tanzu Hub 10.2

Resolution

Check if the environment is using a proxy or is airgapped. At this time, the trivy database download requires internet access with a Harbor tile installed to host the database. The product team is working on this limitation in the 10.2.1 patch.