NSX Manager reports “Upgrade in progress” after restoring backup taken after upgrade finished

Products

• VMware NSX

Issue/Introduction

• Complete upgrade to 4.2.x then restored NSX Manager after completing upgrade
  
  
• After performing an NSX Manager restore, the restore workflow completes, but when you log in to the NSX UI, you see an "An upgrade is currently in progress" notification.
  
   
  
  
• The upgrade appears to be left in a pending/paused state even after the restore finishes, and users cannot continue the normal upgrade flow from the UI.
  When opened "System" -> "Upgrade" in NSX Manager after restore, the "host upgrade" and "preparation for upgrade" is checked as completed, but "NSX Manager upgrade" is not checked.
  
  
• Looking at the /var/log/upgrade-coordinator/upgrade-coordinator.log on the NSX Manager, we see that the MP componentUpgradeStatus is paused at 70%.
  
  

  ####-##-##T##:##:##.###Z  INFO http-nio-127.0.0.1-7442-exec-1 UpgradeQueryServiceImpl ###### SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] For component type EDGE componentUpgradeStatus is SUCCESS 100%
  ####-##-##T##:##:##.###Z  INFO http-nio-127.0.0.1-7442-exec-1 UpgradeQueryServiceImpl ###### SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] For component type HOST componentUpgradeStatus is SUCCESS 100%
  ####-##-##T##:##:##.###Z  INFO http-nio-127.0.0.1-7442-exec-1 UpgradeQueryServiceImpl ###### SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] For component type MP componentUpgradeStatus is PAUSED 70% <-----!  

 

 

Environment

NSX 4.2.1.X
NSX 4.2.2.X

Cause

A migration control file used by the upgrade logic contained version-matching entries that were too broad and a formatting mistake.
As a result, certain migration tasks were incorrectly considered applicable after a restore of the same minor release build, so the UI continued to report an ongoing upgrade.

Resolution

This issue fixed in NSX 4.2.3

Workaround: 

1. After the restore has completed, invoke the Upgrade Coordinator continue API on the Manager:
   
   POST /api/v1/upgrade/plan?action=continue
   
   
2. Navigate to [System] > [Upgrade] then refresh browser

Additional Information

Note:
If the API request POST /api/v1/upgrade/plan?action=continue fails with the following error, the restoration is not yet complete.

{
  "httpStatus" : "BAD_REQUEST",
  "error_code" : 30955,
  "module_name" : "upgrade-coordinator",
  "error_message" : "Pre-upgrade checks failed for MP: Restore operation is in state: SUSPENDED_FOR_USER_ACTION. Please complete the restore operation before attempting to upgrade." <-----! 
}

Go to [System] > [Backup & Restore] and check if the restore status is "Suspended".
If the restore status is Suspended, select [Show Details], and carefully read the notes before selecting [RESUME] to resume the restore.