BackupManager encountered an exception while taking VAMI backups

Products

• VMware vCenter Server

Issue/Introduction

Unable to take schedule\manual vCenter backups from VAMI.

it fails with an error: "BackupManager encountered an exception"

In var\log\vmware\applmgmt\backup.log, you will find below traces:

2024-11-08T05:24:56.135 [20241108-052346-24026615] [MainProcess:PID-52101] [SftpStorageIOLib::_authorizeLocalHost:SftpStorageIOLib.py:168] INFO: Authorizing
localhost with command sshpass -f /tmp/tmp8c6_po4q ssh-copy-id -i /root/.ssh/id_rsa.pub -p 22 -o UserKnownHostsFile=/root/.ssh/br_temp_known_hosts vcenter_us
[email protected]
2024-11-08T05:24:56.223 [20241108-052346-24026615] [MainProcess:PID-52101] [SftpStorageIOLib::_authorizeLocalHost:SftpStorageIOLib.py:174] ERROR: Failed to a
uthorize local host with command: sshpass -f /tmp/tmp8c6_po4q ssh-copy-id -i /root/.ssh/id_rsa.pub -p 22 -o UserKnownHostsFile=/root/.ssh/br_temp_known_hosts
 [email protected]. Will proceed with backup through curl

2024-11-08T05:24:56.224 [20241108-052346-24026615] [MainProcess:PID-52101] [SftpStorageIOLib::_readRemoteRoot:SftpStorageIOLib.py:145] INFO: Executing comman
d to get remote root: ssh -q -p 22 -i /root/.ssh/id_rsa -o UserKnownHostsFile=/root/.ssh/br_temp_known_hosts 'vcenter_user'@'10.xx.xx.x7' echo ~

2024-11-08T05:25:40.576 [20241108-052346-24026615] [MainProcess:PID-52101] [SftpStorageIOLib::mkdir:SftpStorageIOLib.py:350] WARNING: Parent dir sftp://<SFTP server address>/sftp_data create failed. RC: 21 Err: curl: (21) mkdir command failed: Operation failed

2024-11-08T05:26:04.618 [20241108-052346-24026615] [ComponentScriptsBackup:PID-62035] [Proc::GetProcsStatus:Proc.py:345] ERROR: Process returncode is -13, but expected exit codes are [0].
2024-11-08T05:26:04.619 [20241108-052346-24026615] [ComponentScriptsBackup:PID-62035] [Proc::GetProcsStatus:Proc.py:327] ERROR: rc: 1, stderr: Traceback (most recent call last):
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/plugins/../util/Calculate.py", line 59, in <module>
    main(sys.argv[1], sys.argv[2], sys.argv[3])
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/plugins/../util/Calculate.py", line 46, in main
    stdout_obj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

 

 

Environment

VMware vCenter server

Cause

If there is a disk space issue during data transfer, you should see "BrokenPipeError".
Please Note that "BrokenPipeError" error is seen for data transfer due to any reason.

Backup space unavailibility is one of those reasons. 

Data transfer for SFTP backups happens over ssh tunnel or with curl commands. 

From the logs, we can see that the mkdir command was failed.

Resolution

There is not enough space left on the backup location in sftp server.

Please login to sftp server and check the location where backup is getting saved.

Partition name, you can find from logs:

<SFTP server address>/<folder_name>

Run df -h if its linux server and check the utilization:

Filesystem             Size  Used Avail Use% Mounted on
/dev/mapper/rhel-root   50G  135M   50G   1% /
devtmpfs               5.9G     0  5.9G   0% /dev
tmpfs                  5.9G     0  5.9G   0% /dev/shm
tmpfs                  5.9G  613M  5.3G  11% /run
tmpfs                  5.9G     0  5.9G   0% /sys/fs/cgroup
/dev/mapper/rhel-usr    15G  3.4G   12G  23% /usr
/dev/sda2              5.0G  164M  4.9G   4% /boot
/dev/mapper/vg01-lv01  200G  200G  728M 100% /sftp_data
/dev/mapper/rhel-var    10G  242M  9.8G   3% /var
/dev/mapper/rhel-home   25G   33M   25G   1% /home
/dev/mapper/rhel-tmp    15G   33M   15G   1% /tmp
tmpfs                  1.2G   16K  1.2G   1% /run/user/42
tmpfs                  1.2G     0  1.2G   0% /run/user/1002

 

Cleanup space or add more space to the configured partition and reinitiate the backup.

Backup will be successful.

Additional Information

Please invesitgate storage capacity from the sftp server end.