Backup configuration for SDDC manager is failing - "Invalid parameter: validation failed for directory path </path/to/directory> on server <IP_or_FQDN>"

Products

• VMware SDDC Manager
• VMware Cloud Foundation 4.x

Issue/Introduction

• While attempting to add a backup SFTP target for the SDDC Manager via the UI:
  

  Error Message:
  
  "Invalid parameter: validation failed for directory path </path/to/directory>" on server <IP_or_FQDN>. Ensure the backup directory is intact and the SFTP server has write permissions on the backup path.

• We may still be able to read and write to the SFTP server when connecting to it directly from the SDDC Manager
  
  
  
• From /var/log/vmware/vcf/operationsmanager.log, we see below mentioned error
  
  

  YYYY-MM-DDThh:mm:ss DEBUG [vcf_om,022c14a0bc064510,ef9a] [c.v.e.s.c.u.InMemoryHostKeyRepository,http-nio-127.0.0.1-7300-exec-1] Key <KEY-STRING>= of type ecdsa-sha2-nistp256 for host <target-backup-server> not found in [KeySpec(host=<target-backup-server>, port=Optional[22], keyType=ssh-rsa, key=<KEY-STRING>)]
  YYYY-MM-DDThh:mm:ss ERROR [vcf_om,022c14a0bc064510,ef9a] [c.v.evo.sddc.common.util.SshUtil,http-nio-127.0.0.1-7300-exec-1] Unable to create jsch CLI session:
  com.jcraft.jsch.JSchException: reject HostKey: <target-backup-server>

Environment

VMware vCloud Foundation 4.5.x

Cause

The root cause of this issue is related to a problem in negotiating SSH keys with OpenSSH server versions 8:8 and higher. 

Resolution

This issue is resolved in VMware cloud foundation 5.1.1 For more information visit VMware Cloud Foundation 5.1.1 Release Notes

 

Workaround: -

You will need to manually fetch the SSH RSA fingerprint from the backup server and register it through the SDDC Manager API.

1. Fetch SSH-RSA SHA256 Fingerprint of the Backup Server:

   Execute the following command to retrieve the fingerprint of the SSH server:

    

   # ssh-keygen -lf <(ssh-keyscan -t ssh-rsa 22 <ip-or-fqdn-of-the-server> 2>/dev/null)

   The output should display the fingerprint in the following format:

    

   [OUTPUT_HERE]

2. Register the Backup Server Using the VCF API:

   • Log in to the SDDC Manager UI.
   • Navigate to "Developer Center" → "API Explorer" → "APIs for Managing Backups and Restore".
   • Select the PATCH method for the /v1/system/backup-configuration endpoint.

3. Use the Following JSON to Register the Backup Server:

   Populate the required fields with your server's details, including the fingerprint you fetched earlier.

    

   backup_config.json
   {
       "backupLocations": [ {
           "server": "<backup-server-ip-or-fqdn>",
           "port": 22,
           "protocol": "SFTP",
           "directoryPath": "<directory-path>",
           "sshFingerprint": "[OUTPUT_HERE]",
           "username": "<user>",
           "password": "<password>",
       } ],
       "encryption": {
           "passphrase": "<passphrase>"
       }
   }

4. Execute the API:

   Once you've populated the required fields, execute the API call. Wait for the Backup-Configuration task to complete successfully. You can track the task status in the Tasks panel within the SDDC Manager UI.

5. Verify Backup Configuration:

   To verify that your backup configuration is working, trigger a backup manually via the SDDC Manager UI or through the API.