"Inconsistent" state is reported after enabling automatic database failover mode in Cloud Director 10.4.1.1

Products

• VMware Cloud Director

Issue/Introduction

• The database reports 'Indeterminate' state when trying to enable Automatic failover on the database cluster.
• The API /api/1.0.0/nodes/failover/status reports that repmgrd is "NOT RUNNING".
• The Cloud Director environment is an upgraded one and not a fresh install.
• After running the command 'journalctl -xe' you see the following in the output:

[ERROR] provided configuration file "/opt/vmware/vpostgres/10/etc/repmgr.conf" not found

• Within the file /usr/lib/systemd/system/repmgrd-pg14.service you can see that the Environment and ExecStart parameters reference postgres 10 directories and not postgres 14.

Environment

VMware Cloud Director 10.4.1.x

Cause

This issue occurs on upgraded systems as the postgres 10 repmgr RPM is reinstalled during the upgrade process. This reinstall is expected, but a logic issue occurs which results in the /usr/lib/systemd/system/repmgrd-pg14.service file incorrectly referencing postgres 10 directories.

Resolution

This matter is resolved in Cloud Director 10.5.1 which is available for download here.

 

The following workaround is available if installation of the available patch is not immediately possible.

1. Backup Cloud Director as outlined here.
2. Login to the Cloud Director cells as root and remove the rpm package.

rpm -e repmgr
 

3. Recreate the symlink to point to the pg14 version of repmgrd:

ln -s /usr/lib/systemd/system/repmgrd-pg14.service /usr/lib/systemd/system/repmgrd.service
 

4. Restart repmgrd

systemctl restart repmgrd.service