Upgrade from vRA 7.3 to 7.5 fails during script execution, /etc/bootstrap/postupdate.d/09-90-prepare-psql: Failed with exit code 256

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Similar entries below found in /opt/vmware/var/log/vami/updatecli.log:

res=1
echo 'Script /etc/bootstrap/postupdate.d/09-90-prepare-psql failed, error status 1'
exit 1
rm -f /tmp/postupdate-err-log
exit 1
trapfunc
excode=1
test 1 -gt 0
vami_update_msg set post-install 'Post-install: failed'
test -x /usr/sbin/vami-update-msg
/usr/sbin/vami-update-msg set post-install 'Post-install: failed'
sleep 1
test 1 -gt 0 -o 0 -gt 0
vami_update_msg set update-status 'Update failed (code 0-1). Check logs in /opt/vmware/var/log/vami or retry update later.'
test -x /usr/sbin/vami-update-msg
/usr/sbin/vami-update-msg set update-status 'Update failed (code 0-1). Check logs in /opt/vmware/var/log/vami or retry update later.'
exit
03/02/2019 00:48:28 [ERROR] Failed with exit code 256

• Similar entries below found in /var/log/bootstrap/postupdate.log

2019-02-03 00:48:26 PSQL perform base backup for this replica
Located file '/etc/vr/psql/db.env' to preset current environment settings for the database.
[2019-02-03 00:48:26] [root] [INFO] Default settings are overridden!
[2019-02-03 00:48:26] [root] [INFO] <psql-set-replica> Replication user is: <vcac_replication>
[2019-02-03 00:48:26] [root] [INFO] <psql-set-replica> Alter current Postgres user role for replication
[2019-02-03 00:48:26] [root] [INFO] <psql-set-replica>: locking psql-set-replica command
[psql]: Service vpostgres stopping...
Stopping VMware vPostgres: pg_ctl: directory "/var/vmware/vpostgres/current/pgdata" is not a database cluster directory
Last login: Sun Feb 3 00:48:08 UTC 2019
pg_ctl: directory "/var/vmware/vpostgres/current/pgdata" is not a database cluster directory
Last login: Sun Feb 3 00:48:26 UTC 2019
[psql]: Service vpostgres stopping DONE!
cp: cannot stat `/var/vmware/vpostgres/current/pgdata/postmaster.opts': No such file or directory
cp: cannot stat `/var/vmware/vpostgres/current/pgdata/pg_log': No such file or directory
[2019-02-03 00:48:26] [root] [INFO] [psql-util]: Reading confirmation, confirm mode is set to:
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
transaction log start point: 460/99000028 on timeline 1
pg_basebackup: starting background WAL receiver
pg_basebackup: could not get transaction log end position from server: ERROR: could not open file "./postgresql.conf_backup.Nov-28-2018": Permission denied
Last login: Sun Feb 3 00:48:26 UTC 2019
Script /etc/bootstrap/postupdate.d/09-90-prepare-psql failed, error status 1
pg_basebackup: replication stream was terminated before stop point

 

Environment

VMware vRealize Automation 7.x

Cause

The failure is generated due to a lack of permissions on postgresql.conf_backup.<date> file.

This was a manual backup created file by the user while logged in as root. Found that it had root permissions:

• -rw------- 1 root root 21700 Nov 28 09:20 postgresql.conf_backup.<date>

Whereas the postgresql.conf file has owner permissions as 'postgres users'

Resolution

To resolve the problem, follow the steps below:

1. Roll back all nodes to a stage prior to the upgrade attempt
2. Reset the permission to 'postgres users'

chown postgres:users postgresql.conf_backup.<date>

3. Results:

-rw------- 1 postgres users 21700 Nov 28 09:20 postgresql.conf_backup.<date>

4. Attempt the upgrade again

Additional Information

Impact/Risks:
Manually created back up copies of postgres files can interrupt the pg_basebackup script process due to incorrect permissions.