Unable to run database backup script after upgrading to version 10.3.3

Products

• VMware Cloud Director

Issue/Introduction

Version 10.3 of VMware Cloud Director implements a new method of managing certificates. On occasion, the conversion from the old certificate format to the new certificate format fails during the upgrade process. The new certificate format uses a .pem and .key file for the HTTP and ConsoleProxy endpoints, each with a unique path. The create-backup.sh script references these paths, so if the files did not properly convert, the backup script will error out.

Symptoms:

• After upgrading to version 10.3 or later, the VAMI backup and/or backup script fails to execute
• When trying to create a backup from the VAMI, the following error is seen: 
• When running the /opt/vmware/appliance/bin/create-backup.sh script to generate a backup copy of the database, it fails with the following error:
  • "Failed to take a backup of . Check if exists. /opt/vmware/appliance/bin/create-backup.sh: line 101: [: -ne: unary operator expected"
• When the create-backup.sh script fails, the HTTP_CERT_FILE, HTTP_KEY_FILE, CONSOLEPROXY_CERT_FILE, and CONSOLEPROXY_KEY_FILE paths are blank

Environment

VMware Cloud Director 10.x

Cause

This issue occurs because the HTTP and ConsoleProxy certificates failed to convert to the new format during the upgrade to version 10.3. The backup script is unable to locate the .pem and .key files for the certificates and thus fails to complete.

Resolution

To resolve this issue, the HTTP and ConsoleProxy certificates must be re-imported into each cell in the VCD environment. For versions 10.3 and later, this requires a .pem file and a .key file. The .pem file is effectively the certificate, and the .key file is the associated key. This process is detailed in the VCD documentation found in the link below:
https://docs.vmware.com/en/VMware-Cloud-Director/10.3/VMware-Cloud-Director-Install-Configure-Upgrade-Guide/GUID-74296A4D-416F-4D7F-BBD6-BA983AAAD4E7.html#GUID-74296A4D-416F-4D7F-BBD6-BA983AAAD4E7

Workaround:
To workaround this issue, take a backup of the database using the built-in postgres utilities, as shown below:

su postgres
/opt/vmware/vpostgres/current/bin/pg_dump vcloud > dbdump.out

Additional Information

Impact/Risks:
This will prevent the collection of database backups from the command line, as well as appliance backups from the VAMI.