NSX Manager upgrade does not proceed after 2 Manager upgrade completed

Products

• VMware NSX

Issue/Introduction

• Upgrading NSX Manager from version is 3.2.1.x or later. It is rolling upgrade.
• 2 Manager nodes has been upgraded successfully.
• The last node started upgrade but it's in progress forever at 1%.
• In [System] - [Backup and Restore], Upgrade history is loading forever.

Environment

VMware NSX-T Datacenter 3.2
VMware NSX 4.0
VMware NSX 4.1

Cause

Upgrade process is handed over from the orchestrator node to another Manager node when the last Manager node is upgraded.
The hand-over gets stuck forever if the Manager node that the upgrade process is handed over to can't retrieve backup files on the backup server.

Resolution

The issue won't be seen when upgrading to NSX 4.2 or later. If upgrade gets stuck, follow the steps to resume upgrade.

• • Make sure all the NSX Manager can list backup.
  • Make sure login to the backup server is working with the configured credential.
    The KB might be helpful.
    NSX-T Manager Backup task never completes Refer KB:371358
  • Restart Upgrade Coordinator on all the 3 Manager nodes.
    restart the service in NSX CLI restart service install-upgrade
  • Retry the upgrade.