vCenter Server VAMI backups are not getting deleted even after specifying the retention policy

Products

• VMware vCenter Server

Issue/Introduction

• Backup works as expected.
  
  
• vCenter server VAMI backup files on the backup server are not being deleted even though the retention policy has been defined. 
  
  
• There may be similar error messages in backupScheduler.log

YYYY-MM-DDTHH:MM:SS.363 [0] [MainProcess:PID-2226361] [Scheduler::ExecRetentionJob:Scheduler.py:99] ERROR: Failed to delete old backup files from location: backup_location/S_8.0.3.00400_20250127-045905_. Err: Plugin error occurred. ErrCode: 1, Args: ()
  ...
  plugins.StorageIOErrorsLib.PluginError: Plugin error occurred. ErrCode: 1, Args: ()
YYYY-MM-DDTHH:MM.417 [0] [MainProcess:PID-2226361] [SMBClientStorageIOLib::_process_cmd_output:SMBClientStorageIOLib.py:97] ERROR: Command failed with error: NT_STATUS_ACCESS_DENIED
  
  or
  
  
• YYYY-MM-DDTHH:MM:SS [0] [MainProcess:PID-3309### [Scheduler::ExecRetentionJob:Scheduler.py:95] INFO: Removing old backup from location: #########
YYYY-MM-DDTHH:MM:SS [0] [MainProcess:PID-3309###] [FtpStorageIOLib::_process_curl_output:FtpStorageIOLib.py:152] ERROR: sftp cmd failed. RC: 21, Err: curl: (21) rm command failed: Permission denied, Cmd: ['/usr/bin/curl', '--fail', '-u', '<backup username>:****', '--connect-timeout', '120', '--ssl-reqd', '-k', '--noproxy', '*', '--silent', '--show-error', '-Q', 'rm /vmware-backup/vCenter/############', 'sftp:/#######'], stdout:
YYYY-MM-DDTHH:MM:SS [0] [MainProcess:PID-3309###] [Scheduler::ExecRetentionJob:Scheduler.py:99] ERROR: Failed to delete old backup files from location: ###########_. Err: Plugin error occurred. ErrCode: 102, Args: ()
Traceback (most recent call last):
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/Scheduler.py", line 97, in ExecRetentionJob
    dispatch.storageIO.rmdir(relative_loc=fileName)
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/plugins/FtpStorageIOLib.py", line 192, in rmdir
    self.unlink(relative_loc=os.path.join(relative_loc, fname))
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/plugins/FtpStorageIOLib.py", line 187, in unlink
    self._process_curl_output(cmd_res)
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/plugins/FtpStorageIOLib.py", line 161, in _process_curl_output
    raise PluginError(plugins.StorageIOErrorsLib.PluginError: Plugin error occurred. ErrCode: 102, Args: ()
YYYY-MM-DDTHH:MM:SS [0] [MainProcess:PID-3309###] [Scheduler::ExecRetentionJob:Scheduler.py:95] INFO: Removing old backup from location:###########.
YYYY-MM-DDTHH:MM:SS [0] [MainProcess:PID-3309###] [FtpStorageIOLib::_process_curl_output:FtpStorageIOLib.py:152] ERROR: sftp cmd failed. RC: 21, Err: curl (21) rm command failed: Permission denied, Cmd: ['/usr/bin/curl', '--fail', '-u', '<backup username>', '--connect-timeout', '120', '--ssl-reqd', '-k', '--noproxy', '*', '--silent', '--show-error', '-Q', '
rm /vmware-backup/vCenter/##########', 'sftp://##########'], stdout:
YYYY-MM-DDTHH:MM:SS [0] [MainProcess:PID-3309###] [Scheduler::ExecRetentionJob:Scheduler.py:99] ERROR: Failed to delete old backup files from location: ############ Err: Plugin error occurred. ErrCode: 102, Args: ()

Environment

VMware vCenter Server 8.x
VMware vCenter Server 7.x

Cause

The backup user does not have the privileges to make changes or delete files in the backup directory.

Resolution

Make sure the backup user has the required privileges to delete files and folders in the backup directory.