SFTP Backup Registration Fails with SSH Fingerprint Error on SDDC Manager
search cancel

SFTP Backup Registration Fails with SSH Fingerprint Error on SDDC Manager

book

Article ID: 438943

calendar_today

Updated On:

Products

VCF Operations VMware SDDC Manager / VCF Installer

Issue/Introduction

When attempting to register or add a Linux-based SFTP backup server in SDDC Manager, the task fails during validation or registration. This issue is typically encountered during initial VCF 9.0+ deployments or when reconfiguring backups in hardened VCF 5.x environments. Users may see misleading errors regarding directory permissions while the underlying failure is related to cached host key mismatches.
  • The task fails with the error: unable to make directory on the backup server.
  • The API call used for registration fails with: unable to fetch the SSH fingerprint.
  • In the log directory /var/log/vmware/vcf/operationsmanager/operationsmanager.log of SDDC manager, following error is seen:

    YYYY-MM-DD HH:MM:SS.SSS+0000 DEBUG [vcf_om,<hex-value>] [c.v.e.sddc.common.util.SshKeyHelper,pool-#-thread-#] Found ecdsa SSH key for host <sftp-server-fqdn>: {"host":"<sftp-server-fqdn>","keyType":"ecdsa-sha2-nistp256","key":"<key-string>"}
    YYYY-MM-DD HH:MM:SS.SSS+0000 INFO  [vcf_om,<hex-value>] [c.v.v.c.f.p.b.r.v.BackupLocationValidator,pool-#-thread-#] Given SSH fingerprint didn't match any of the fingerprints on server <sftp-server-fqdn>
 

Environment

VMware Cloud Foundation 5.x

VMware Cloud Foundation 9.x

Cause

  • Unsupported Key Type: VMware Cloud Foundation 9.0 and higher require an ECDSA-based SSH fingerprint for SFTP backup configuration. RSA keys are not supported for fingerprint authentication in these releases. This is separate from the SSH host key algorithms the backup server must advertise to complete the connection.
  • Incorrect File Permissions: The known_hosts files on the SDDC Manager have incorrect permissions or ownership, preventing the system from writing or updating the fingerprints even after a successful manual injection attempt.
  • Fingerprint Mismatch: If the SFTP server was re-imaged or its keys regenerated, the SDDC Manager’s cached entry in its internal trust store will reject the session.

Resolution

Confirm the backup server presents an ECDSA key, correct the permissions on the relevant known_hosts files, then register the key. All steps run on the SDDC Manager over SSH as root.

Step 1: Verify the ECDSA fingerprint

Most SSH clients show the RSA host key fingerprint by default. VCF uses the ECDSA fingerprint. An RSA fingerprint never matches the value VCF shows, and that difference does not indicate a fault.

  1. Get the backup server's ECDSA fingerprint:

    ssh-keyscan -t ecdsa <sftp-server-fqdn> | ssh-keygen -lf -
    
  2. Compare the returned value against the fingerprint shown in the backup configuration interface.

  3. If the command returns nothing, the backup server is not presenting an ECDSA host key. Configure the server to offer one, then repeat this step. See Additional Information.

  4. If the values match, go to Step 2.

Step 2: Correct file permissions and ownership

Set permissions to 644 and correct ownership on each file.

  1. Root SSH:

    chmod 644 /root/.ssh/known_hosts && chown root:root /root/.ssh/known_hosts
    
  2. Common Services:

    chmod 644 /etc/vmware/vcf/commonsvcs/known_hosts && chown vcf_commonsvcs:vcf /etc/vmware/vcf/commonsvcs/known_hosts
    
  3. VCF user:

    chmod 644 /home/vcf/.ssh/known_hosts && chown vcf:vcf /home/vcf/.ssh/known_hosts
    
  4. Default hosts:

    chmod 644 /opt/vmware/vcf/commonsvcs/defaults/hosts/known_hosts && chown vcf_commonsvcs:vcf /opt/vmware/vcf/commonsvcs/defaults/hosts/known_hosts
    

Step 3: Refresh the cached host keys

This clears stale entries, for example after the backup server was re-imaged or its keys were regenerated. It does not add a server that was never registered.

  1. Refresh the cached keys:

    curl -X POST 'http://localhost/appliancemanager/ssh/knownHosts/refresh'
    
  2. Check whether the backup server is listed:

    curl -s 'http://localhost/appliancemanager/ssh/knownHosts'
    
  3. If the backup server is listed, go to Step 6. If it is absent, go to Step 4.

Step 4: Get the ECDSA host key

  1. Retrieve the key from the backup server:

    ssh-keyscan -t ecdsa <sftp-server-fqdn>
    
  2. Read the output. It returns one key line in the form hostname keytype key, preceded by a comment line that begins with a hash character and can be ignored. The key line looks like this, with the key shown masked:

    sftp.rainpole.io ecdsa-sha2-nistp256 ########################################################################
    
  3. Copy the third field only. Do not include the hostname, the key type, or the comment line. This value is referred to below as the key string.

Step 5: Register the host key

Use either form below. Both send the same request to the same endpoint. Use the curl command at the shell, or the JSON body in an API client.

Replace each placeholder with your value, deleting the angle brackets along with the placeholder text. The finished request contains no angle brackets.

PlaceholderReplace with
<sftp-server-fqdn>The backup server's fully qualified domain name. Do not use a short hostname. It must match the address used in the backup configuration.
<key-string>The third field only, copied in Step 4.
  1. Shell form:

    curl -i -X POST 'http://localhost:7100/appliancemanager/ssh/knownHosts' -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{"knownHosts":[{"host":"<sftp-server-fqdn>","port":22,"keyType":"ecdsa-sha2-nistp256","key":"<key-string>"}]}'
    
  2. API client form. Send a POST to http://localhost:7100/appliancemanager/ssh/knownHosts with headers Content-Type: application/json and Accept: application/json, and this request body:

    {
      "knownHosts": [
        {
          "host": "<sftp-server-fqdn>",
          "port": 22,
          "keyType": "ecdsa-sha2-nistp256",
          "key": "<key-string>"
        }
      ]
    }
    
  3. If the request returns HTTP 400, the payload is malformed. Check for angle brackets left in a value, a key string that still includes the hostname or key type, or a short hostname used in place of the fully qualified domain name.

Step 6: Confirm and retry

  1. Confirm the entry was accepted:

    curl -s 'http://localhost/appliancemanager/ssh/knownHosts'
    

    The backup server appears in the returned list.

  2. Retry the backup configuration.

  3. If registration still fails, see Additional Information.

Additional Information

Two separate SSH requirements apply to SFTP backup targets, and the error messages do not clearly distinguish them.

The fingerprint requirement is covered by this article. VCF uses the backup server's ECDSA host key to derive the fingerprint.

Separately, the backup server must advertise at least one of rsa-sha2-512 or rsa-sha2-256, together with at least one of ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, or ecdsa-sha2-nistp521. A server offering only the legacy ssh-rsa algorithm alongside ecdsa-sha2-nistp256 provides a valid ECDSA fingerprint but cannot complete SSH negotiation. The resulting failure can resemble a fingerprint problem.

To test this from the SDDC Manager:

ssh -o HostKeyAlgorithms=rsa-sha2-512,rsa-sha2-256 -p 22 <sftp-server-fqdn>

A response of Unable to negotiate ... no matching host key type found indicates this condition. The fix is applied to the backup server rather than to VCF. See KB 441702, SDDC Manager Backup Configuration Fails: https://knowledge.broadcom.com/external/article/441702