Unable to back out of NSX upgrade once proceeded to pre-check phase

Products

• VMware NSX

Issue/Introduction

1. You uploaded the mub file >> prepared upgrade co-ordinator >> ran pre-checks and then decide to back out of the upgrade for any reason, but you are unable to back out and NSX notifications pane states that upgrade is in progress as in below screenshot. 



2. Running the below command shows "Upgrade is not in progress" on any of the manager nodes as in below sample.

get upgrade progress-status

Sample output of the above command:

Manager-node> get upgrade progress-status
Date and time
Upgrade is not in progress

Environment

VMware NSX

Cause

Upgrade-coordinator is already upgraded and hence unable to back out from the upgrade

Resolution

Rollback the upgrade coordinator by running the following commands from the admin shell on the orchestrator node:

1. set debug
2. rollback upgrade-coordinator

Post running the above commands, you might have to wait for a couple of minutes to get the upgrade pane visible in the NSX Manager UI.

Additional Information

You can identify the orchestrator node by running the below command on any of the manager and verify which node is the service enabled on (Enabled on IP in the below command output is the orchestrator node IP):

get service install-upgrade

Sample output of the above command:

Manager-node> get service install-upgrade
Date and Time
Service name: install-upgrade
Service state: running
Enabled on: <IP of the orchestrator node>

Note: Please run the above command on all the managers and make sure that all the mangers are reporting the same orchestrator node.