VMware NSX "Backup" to SFTP server fails with FIPS violation.
Article ID: 377909
Updated On:
Products
Issue/Introduction
VMware NSX backup to a SFTP server fails with following error.
"Possible FIPS violation during a backup: Error negotiating with remote host: Unable to negotiate with <sftp-server-IP> port 22: no matching host key type found. Their offer: rsa-sha2-512,rsa-sha2-256,ssh-rsa"
api_server.log at /var/log/nvpapi/ will report following error
2024-09-17T08:15:16.323Z napi.root.node.backup_restore INFO Copying cluster backup file, location: sftp://<sftp-server-fqdn>/scpbackup/####/####/NSX//cluster-node-backups/4.1.2.0.0.####-####-###-###-####-#####-<SFTP-Server-IP>/backup-##-##-##/cluster_backup-#####-####-####-###-######-<SFTP-Server-IP>-nsx-ufo-backup-restore.tar
2024-09-17T08:15:16.758Z napi.root.node.backup_restore ERROR Cluster backup file copy operation failed due to 400 Bad RequestContent-Type: application/jsonContent-Length: 235
{"error_code": 36209, "error_message": "Error negotiating with remote host: Unable to negotiate with <SFTP-Server-IP> port 22: no matching host key type found. Their offer: rsa-sha2-512,rsa-sha2-256,ssh-rsa", "module_name": "node-services"}2024-09-17T08:15:21.509Z nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824, local endpoint: 127.0.0.1:39336) Close CloseReason=LOCAL_CLOSE destroy 12024-09-17T08:15:21.509Z nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824, local endpoint: 127.0.0.1:39336) shutdown <gevent._socket3.socket at 0x7e1f401ecea0 object, fd=37, family=2, type=1, proto=0>2024-09-17T08:15:21.509Z nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824) Close done2024-09-17T08:15:21.510Z nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824) Close CloseReason=NETWORK_ERROR destroy 02024-09-17T08:15:21.510Z nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824) Close done2024-09-17T08:15:21.511Z napi.root.node.backup_restore ERROR REPEATS: 1 repeats in 4 sec: Cluster backup file copy operation failed due to 400 Bad Request
Environment
VMware NSX
Cause
The failure happens due to differences in the supported host key type between NSX Manager and the SFTP server. In the above log excerpt the backup server is supporting "rsa-sha2-512,rsa-sha2-256,ssh-rsa" and does not have "ecdsa" which NSX Manager is looking for.
Resolution
Ensure that the sftp servers sshd_config have "ecdsa" enabled so that both the NSX Manager and SFTP server can agree on negotiation.
Additional Information
To identify the supported host key algorithm supported by NSX Manager
Run the following command from a ssh session to the NSX Manager and the ones offered by the SFTP server
#ssh -vvv user@<backupserver-IP/FQDN> 22
Example output of the above command
The host key algorithm proposed by the NSX-T Manager
debug2: local client KEXINIT proposaldebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected]debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsadebug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],[email protected]debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],[email protected]
Host key algorithm proposed by sftp server.
debug2: peer server KEXINIT proposaldebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
The negotiation happen on ecdsa-sha2-nistp256
debug1: kex: algorithm: curve25519-sha256debug1: kex: host key algorithm: ecdsa-sha2-nistp256 <--negotiate
Feedback
