vCenter server filebased backups are failing for the SFTP protocol.

Products

• VMware vCenter Server

Issue/Introduction

• SFTP VAMI backup fails with error "Failed to create backup directory on backup server".
• Both Scheduled and Manual backups fail with the same error.
• Intermittently, the scheduled backup may work one time and then a manual backup will fail immediately after.

Environment

vCenter 9.x

Cause

vCenter SFTP backup is hitting a PerSourcePenalty issue, this can be confirmed by checking /var/log/message on the SFTP server and observing the following error:

SFTP-FQDN sshd-session[xxxxx]: Unable to negotiate with vCenterIPAddress port 60796: no matching host key type found. Their offer: [email protected]
[preauth]
SFTP-FQDN sshd-session[xxxxx]: Connection closed by authenticating user SFTP-FQDN vCenterIPAdress port 60798 [preauth]

Resolution

• Add the vCenter's IP to the OpenSSH penalty exemption list and reload sshd:
  PerSourcePenaltyExemptList vCenterIPAddress/32
• Alternatively, if there is no use case in place for Penalties, it is possible to disable it globally (prior to OpenSSH 9.8 behavior) 
  PerSourcePenalties no  

Additional Information

This issue will be addressed in a future vCenter release