VMware NSX Backup to SFTP server fails with FIPS violation.
search cancel

VMware NSX Backup to SFTP server fails with FIPS violation.

book

Article ID: 377909

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • VMware NSX backup to a SFTP server fails with following error.

"Possible FIPS violation during a backup: Error negotiating with remote host: Unable to negotiate with <sftp-server-IP> port 22: no matching host key type found. Their offer: rsa-sha2-512,rsa-sha2-256,ssh-rsa"

  • On NSX Manager, api_server.log at /var/log/nvpapi/ will report following error

napi.root.node.backup_restore INFO Copying cluster backup file, location: sftp://<sftp-server-fqdn>/scpbackup/####/####/NSX//cluster-node-backups/4.1.2.0.0.####-####-###-###-####-#####-<SFTP-Server-IP>/backup-##-##-##/cluster_backup-#####-####-####-###-######-<SFTP-Server-IP>-nsx-ufo-backup-restore.tar

napi.root.node.backup_restore ERROR Cluster backup file copy operation failed due to 400 Bad Request
Content-Type: application/json
Content-Length: 235

{"error_code": 36209, "error_message": "Error negotiating with remote host: Unable to negotiate with <SFTP-Server-IP> port 22: no matching host key type found. Their offer: rsa-sha2-512,rsa-sha2-256,ssh-rsa", "module_name": "node-services"}
nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824, local endpoint: 127.0.0.1:39336) Close CloseReason=LOCAL_CLOSE destroy 1
nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824, local endpoint: 127.0.0.1:39336) shutdown <gevent._socket3.socket at 0x7e1f401ecea0 object, fd=37, family=2, type=1, proto=0>
nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824) Close done
nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824) Close CloseReason=NETWORK_ERROR destroy 0
nsxrpc INFO NsxRpcConnection (0x7e1f3a91c5e0 remote endpoint: tcp://127.0.0.1:9824) Close done
napi.root.node.backup_restore ERROR REPEATS: 1 repeats in 4 sec: Cluster backup file copy operation failed due to 400 Bad Request

Environment

VMware NSX 4.2.x

Cause

  • The failure happens due to differences in the supported host key type between NSX Manager and the SFTP server.
  • In the above log excerpt the backup server is supporting "rsa-sha2-512,rsa-sha2-256,ssh-rsa" and does not have "ecdsa-sha2-nistp256" which NSX Manager is looking for

Resolution

The issue has been resolved in NSX version 9.0 and later versions.

Workaround:
Ensure that the sftp servers sshd_config have "ecdsa-sha2-nistp256" enabled so that both the NSX Manager and SFTP server can agree on negotiation.

 

Additional Information

  • To identify the supported host key algorithm supported by NSX Manager, run the following command from NSX Manager:

#ssh -vvv user@<backupserver-IP/FQDN> 22

  • Example output of the above command
    • The host key algorithm proposed by the NSX-T Manager

debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected]
debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],[email protected]
debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],[email protected]

    • Host key algorithm proposed by sftp server.

debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519

    • The negotiation happen on ecdsa-sha2-nistp256

debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256 <--negotiate