Autoscaler fails to restart after a bbr backup

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Symptoms:
When running bbr backup autoscaler does not restart after the backup is completed.

1 error occurred:
error 1:
2 errors occurred:
error 1:
Error attempting to run post-backup-unlock for job lock-unlock-pcf-autoscaling on backup-prepare/4be5ea36-8ac1-4596-b375-4666772b4635: + exec - exit code 1
error 2:
Error attempting to run post-backup-unlock for job bbr-usage-servicedb on backup-prepare/4be5ea36-8ac1-4596-b375-4666772b4635: + exec /var/vcap/jobs/bbr-usage-servicedb/bin/unlock

Environment

Cause

When running bbr backup various components lock, including the Cloud Controller API (CAPI) and autoscaler. After the backup completes, components unlock in a pre-determined order, starting with CAPI and then autoscaler. However, CAPI's unlock script reports being finished before CAPI is fully available, so when autoscaler attempts to unlock, it receives an error when attempting to communicate with CAPI, causing the autoscaler unlock script to fail.

Resolution

If bbr backup fails, run bbr backup-cleanup after the failure, which will unlock autoscaler. bbr backup-cleanup is idempotent, so it can safely be run after every backup, whether successful or not.

The backup artifact produced during the original backup should still be valid.