The Postgres pod in TCA Manager fail to start due to an expired certificate

Products

• VMware Telco Cloud Automation

Issue/Introduction

TCA pods couldn't start as the postgres pod was down with expired certificates. 

Environment

3.x

Cause

TCA Pods cannot connect to postgresinstance when the certificates have expired

kubectl get pods -A | grep -i postgres

postgres-operator-system postgres-operator-57XXXXXXXX-XXXXX 1/1 Running 0 21m
tca-mgr postgres-0         4/5 Running 6 (4m33s ago) 21m
tca-mgr postgres-monitor-0 3/4 CrashLoopBackOff 8 (3m23s ago) 21m

 

2024-10-16T07:46:23.682Z INFO postgresinstance failed to connect to `host=postgres-monitor-0.postgres-agent.tca-mgr.svc.cluster.local user=autoctl_node database=pg_auto_failover`: dial error (dial tcp 100.xxx.xxx.xxx:5432: connect: connection refused)

2024-10-16T07:46:28.690Z INFO postgresinstance failed to connect to `host=postgres-monitor-0.postgres-agent.tca-mgr.svc.cluster.local user=autoctl_node database=pg_auto_failover`: tls error (x509: certificate has expired or is not yet valid: current time 2024-10-16T07:46:28Z is after 2024-10-09T14:31:50Z)

Resolution

Below are the steps to renew postgres certificates :

kubectl delete secret -n postgres-operator-system postgres-operator-webhook-server-cert

[For TCA-CP]

kubectl delete secret -n tca-cp-cn postgres-internal-ssl-secret
kubectl delete secret -n tca-cp-cn postgres-metrics-tls-secret

[For TCA-M]

kubectl delete secret -n tca-mgr postgres-internal-ssl-secret
kubectl delete secret -n tca-mgr postgres-metrics-tls-secret

Post deleting the secret, please reboot the appliances.