Scan job failed with error message "tls: failed to verify certificate: x509: certificate has expired or is not yet valid" in the step-metadata-store-plugin container log

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

- Workload is not in a healthy status and source-scanner is in READY=False status. The Messages section shows container step-metadata-store-plugin. error: scan job failed.

$ tanzu apps workload get -n dev sample-app
📡 Overview
......
💾 Source
......
📦 Supply Chain
   name:   source-test-scan-to-url

   RESOURCE           READY   HEALTHY   TIME   OUTPUT
   source-provider    True    True      51m    GitRepository/sample-app
   source-tester      True    True      51m    Runnable/sample-app
   source-scanner     False   False     0s     SourceScan/sample-app
......
💬 Messages
   Workload [HealthyConditionRule]:   condition status: False, message: Scan job failed: container step-metadata-store-plugin. error: scan job failed. logs:
   ... t-format api-json --path /path/to/file.json 1>out.txt 2>error.txt (* Output will be redirected to out.txt and error will be redirected to error.txt .)
......
🛶 Pods
   NAME                                             READY   STATUS      RESTARTS   AGE
   scan-sample-app-gcnx6-pod   1/9     NotReady    0          24s
   sample-app-252mm-test-pod   0/1     Completed   0          51m

 - step-metadata-store-plugin container log displays the error message - tls: failed to verify certificate: x509: certificate has expired or is not yet valid

$ kubectl logs -n dev scan-sample-app-pw9fr-pod -c step-metadata-store-plugin
Flag --cyclonedxtype has been deprecated, will be removed in version "v2.0.0". Use "input-format" instead.
✖  Error: Post "https://metadata-store.abc/api/sourceReport?": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2023-08-24T07:39:17Z is after 2023-08-07T02:54:21Z

Environment

Product Version: 1.5

Resolution

As documented in Multicluster setup for Supply Chain Security Tools - Store, it's necessary to copy SCST - Store CA certificate from the View cluster and then apply it to the Build cluster.
Use below commands to make sure the contents of ca.crt in both View & Build cluster match. If there is contents mismatch, retrieve the ca.crt from the View cluster again and re-configure the secret in the Build cluster. This action should be able to make scan job pass.

# On view Cluster
$ kubectl get secret -n metadata-store ingress-cert -o json | jq -r ".data.\"ca.crt\""
LS0tLS1CRUdJTiBDRVJU......JRklDQVRFLS0tLS0K
 
# On build Cluster
$ kubectl -n metadata-store-secrets get secret store-ca-cert -o json | jq -r ".data.\"ca.crt\""
LS0tLS1CRUdJTiBDRVJU......JRklDQVRFLS0tLS0K