Troubleshooting NSX Backup and Restore Failures

Products

VMware NSX

Issue/Introduction

When troubleshooting NSX Backup and Restore failures, a specific set of data must be gathered at the time of the event. This article details what documentation is required and how to gather it prior to opening a support request with Broadcom.

Environment

VMware NSX 4.x
VMware NSX-T Data Center 3.x

Resolution

Ensure the backup server is a supported software and a supported version. In addition, be sure the ciphers the SFTP server supports match the list of supported ciphers for the NSX version in use.

NSX supports specific SFTP servers and software:

3.x - Configure Backups
4.x - Configure Backups

SSH finger print of a remote server - This is used to ensure the NSX Manager can authenticate properly with the correct SFTP server.

3.x - Remote server
4.x - Remote server

Restore Documentation

3.x - Restore a Backup
4.x - Restore a Backup

Pertinent Log Files (for backup and restore operations)

NSX Manager

/var/log/syslog
/var/log/proton/nsxapi.log
/var/log/nvpapi/api_server.log

Known issues with Backup or Restore operations:

Additional Information

Backup operations

Have no impact on the manager or the datapath functionality of the NSX environment.
Require IP address/FQDN of backup server
Require SFTP server username/password
Require a backup passphrase (backups will not begin if no passphrase is provided)

Restore operations

Require a new NSX Manager appliance to be deployed with the same FQDN and IP address of the Manager Node the backup came from
The new appliance MUST be of the same version/build number of the original backup
- Example: /home/localadmin/backup/NSX/cluster-node-backups/3.2.3.1.0.22104638-e72c1d42-####-####-####-377a470eb2c6-##.##.##.##/backup-2023-11-22T22_22_00UTC/
- Explained: /<backup>/<server>/<directory>/<name>/cluster-node-backups/<NSX-Manager-Node-Version>-<Manager Node ID>-<Manager IP Address>/backup-<YYYY>-<MM>-<DD>T<HH>_<MM>_<SS>UTC/

Appliance build number is part of the backup folder structure

What to expect during a restore operation

The manager may reboot multiple times during the restore
The GUI may become inaccessible multiple times during the restore
Do not manually reboot or interrupt the restore process. Unless the GUI reports the restore has failed, it is moving forward
If a restore file has different passwords than the newly deployed OVA, access to the manager's interface may be lost until the original passwords are used. Ensure the original admin and root user passwords are available as well as the encryption passphrase of the backup being restored
After a successful restore, the NSX Manager interface and API call responses may be slow. This is due to indexes being built against the database and /repository directory being synchronized across managers

If you are contacting Broadcom support about this issue, please provide the following:

Attempt to perform the backup/restore operation
- Is it a scheduled backup?
- Is it a manual backup?
Recreate the error
- Record all steps necessary to recreate the error
- Record all text of any error messages as it appears in the graphical interface
- Include timestamps of when the recreation began and when the error occurred
Retrieve log bundles from all NSX Managers involved in the backup operation
IP address/FQDN of backup server
SFTP server username
Is the backup server a supported version?
Is the SFTP backup user password active?
Does the backup server have enough disk space?

Handling Log Bundles for offline review with Broadcom support