Backup for VMware Cloud Director cell fails

Products

• VMware Cloud Director

Issue/Introduction

• Creating backup for VMware Cloud Director(i.e. vcd) from management page fails with error:
  
  There was error creating Backup(details)
Unable to create backup: A failure occurred while creating the backup on the primary.
  
  
• /opt/vmware/var/log/vcd/backup-remote.log indicates authentication failure when trying to connect to primary vcd cell:
  
  | DEBUG | Invoking: ssh postgres@<primary cell ip> sudo /opt/vmware/appliance/bin/create-backup.sh
| ERROR | Error running command: CompletedProcess(args='ssh postgres@<primary cell ip> sudo /opt/vmware/appliance/bin/create-backup.sh', returncode=255, stdout='', stderr='Welcome to VMware Cloud Director VA\nPermission denied, please try again.\nPermission denied, please try again.\nReceived disconnect from <primary cell ip> port 22:2: Too many authentication failures\nDisconnected from <primary cell ip> port 22\n')
| ERROR | Unable to create backup.
Traceback (most recent call last):
  File "/opt/vmware/appliance/bin/api/backup.py", line 47, in create_backup
    err = 'A failure occured while creating the backup on the primary.'
  File "/opt/vmware/appliance/bin/api/backup.py", line 25, in run
    raise BackupException(err)
BackupException: A failure occured while creating the backup on the primary.
  
  
• Running below command from vcd command line prompts for postgres password while it's not supposed to ask for password:
  
  sudo -n -u postgres ssh postgres@<primary cell ip> sudo /opt/vmware/appliance/bin/create-backup.sh

Environment

VMware Cloud Director 10.x

Cause

An "AllowUsers" list is explicitly specified in ssh configuration of vcd preventing all other users (including postgres) from ssh to the primary vcd cell and execute backup script.

Resolution

The "AllowUsers" setting needs to be removed from ssh configuration:

1. login to primary vcd cell with root account
2. vi /etc/ssh/sshd_config and comment out "AllowUsers“ line
3. restart ssh service:
   service sshd restart

It's not supported to add "AllowUsers” configuration and adding the configuration could adversely impact certain functionality on vcd.