PostgreSQL/MySQL cluster remains "InProgress", in Data Services Manager, after appliance certificate change

Products

• VMware Data Services Manager

Issue/Introduction

If appliance certificate is changed or expires while a cluster is undergoing rolling update (for example, if it is being scaled or upgraded), in rare occasions the cluster status remains forever "InProgress" and database service becomes inaccessible.

This state is permanent and remediation requires manual intervention.

Environment

VMware Data Services Manager

Cause

If appliance certificate is changed or expires while the cluster is undergoing a rolling update, some newly created workload VMs might get stuck with the wrong certificate (the old one or the expired one) and any subsequent appliance certificate changes will not reach them, so TLS handshake will not be successful for them.

This leads to the cluster staying stuck InProgress and database service becoming inaccessible.

Resolution

If you encounter this issue please contact VMware support for assistance