Error: curl: (79) Could not open directory for reading when performing vCenter VAMI SFTP Backup

Products

• VMware vCenter Server

Issue/Introduction

When attempting a file-based backup of the vCenter Server Appliance (VCSA) via the VAMI using the SFTP protocol, the backup fails during the validation or retention cleanup phase.

You observe the following symptoms:

• The backup itself may complete, but older backups are not deleted based on the retention policy.
• In the backup location, apart from the actual backup folders, multiple folders named backup_####-#### can be seen as well.
• In /var/log/vmware/applmgmt/applmgmt.log, error messages can be found similar to:

  ERROR:plugins.FtpStorageIOLib:sftp cmd failed. RC: 79, Err: curl: (79) Could not open directory for reading: Bad message from SFTP server

• The folder this error is report for consists of twice the actual name for one of the folders seen in the backup location, e.g. instead of backup_<date>-<time> it is called backup_<date>-<time>backup_<date>-<time>
• When manually browsing the FTP using a curl command like this one below, the same issue can be seen:

  # curl -v -u <username>:<password> --connect-timeout 120 --ssl-reqd -k --noproxy '*' 'sftp://<sftp-server-address>/<path-to-vcenter-backups>/'

Environment

• VMware vCenter Server 8.0.x
• Third-party SFTP Server, e.g. Solarwinds SFTP (specifically version 2.2.9.60072)

Cause

This issue occurs because the third-party SFTP server does not properly populate the longname field during directory listing requests. This causes the VCSA appliance management service to misinterpret the folder names (often resulting in duplicated name strings) and fail to read or delete the directories.

Resolution

To resolve this issue, perform either of the following:

1. Contact your SFTP server vendor to investigate why the SFTP service is not properly populating the longname attributes for files and folders.
2. Use a different version or a different SFTP server implementation that adheres to standard SSH/SFTP listing protocols. In reported cases, using an older version of the Solarwinds SFTP server has restored functionality.
3. If retention cleanup continues to fail, manually delete expired backup folders from the SFTP target to prevent storage exhaustion.

Additional Information

To verify if the SFTP server is at fault, run the following command, either from the VCSA shell or from another Linux client system, to manually browse the backup location:

# curl -v -u <username>:<password> --connect-timeout 120 --ssl-reqd -k --noproxy '*' 'sftp://>sftp-server-address</<path-to-vcenter-backups>/'

Next, query the server for file names only:

# curl -l -v -u <username>:<password> --connect-timeout 120 -k --noproxy '*' 'sftp://>sftp-server-address</<path-to-vcenter-backups>/'

If the file names are different in the outputs of the 2 commands, the issue resides with the SFTP server implementation.
Should the issue occur despite both commands providing the exact same file names, contact support.