Scheduler Errand Failing Due To Expired Certificate - panic: tls: failed to verify certificate

Products

• VMware Tanzu Application Service

Issue/Introduction

Scheduler Errand may fail during an upgrade when starting the scheduler-broker application. 

Errand 'deploy-scheduler' completed with error (exit code 1)

In the scheduler-broker logs, the following certificate error will be found:

2025-05-21T12:42:43.864-05:00 [APP/PROC/WEB/0] [ERR] panic: tls: failed to verify certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "rootCA")

This error means that the scheduler is attempting to connect to the database using TLS, but does not trust the certificate authority.

When examining the application environment variable, check the certificate to see if it is expired. Remove the \n and decode it using a site like SSLShopper.

cf env scheduler-broker

VCAP_SERVICES: {
  "p.mysql": [
    {
      ---
      ---                                                                                           
        "tls": {
          "cert": {
            "ca": "-----BEGIN CERTIFICATE----------END CERTIFICATE-----"
          }

Resolution

The scheduler application requires a mysql database connection, which supplies this cert to the broker application. Update the cert in the env variable by unbinding and rebinding the mysql service to the application, then restage the application.

cf unbind-service <app> <service>
cf bind-service <app> <service>
cf restage <app>