NSX-T manager upgrade fails due to error while extracting upgrade bundle
search cancel

NSX-T manager upgrade fails due to error while extracting upgrade bundle

book

Article ID: 322488

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • You are upgrading VMware NSX.
  • The Manager Upgrade Bundle (MUB) file extraction fails with the following error in the UI:
Upgrade-coordinator upgrade failed. Error - Error while extracting upgrade bundle VMware-NSX-upgrade-bundle-4.1.0.2.0.21761691.mub
  • We see the following messages in the NSX-T Manager log: /var/log/upgrade-coordinator/upgrade-coordinator.log
2023-09-07T11:07:47.871Z INFO Thread-93 UpgradeBundleUcHelper 23098 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Failed to extract Upgrade Bundle VMware-NSX-upgrade->bundle-4.1.0.2.0.21761691.mub.
2023-09-07T11:07:18.040Z ERROR Thread-92 UpgradeCoordinatorUpgradeService 23098 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP30040" level="ERROR" subcomp="upgrade-coordinator"] Error while updating upgrade-coordinator due to error Error while extracting upgrade bundle VMware-NSX-upgrade-bundle-4.1.0.2.0.21761691.mub : {1}.
  • On the NSX-T Manager, In the log file /var/log syslog we see below entries which indicates that the UC upgrade failure is due to the creation of an extra file "migration-progress.json"
    2023-10-03T13:28:30.759Z MpPolicyController-manager2 - - - - [NSX] [nsx@6876 comp="nsx-manager" subcomp="upgrade-coordinator"] uc_helper: /image/upgrade-coordinator/ should contain upgrade bundle and eula file: 4102021761691.bundle_status, localized_eula, eula.txt, migration-progress.json, VMware-NSX-upgrade-bundle-4.1.0.2.0.21761691.tar.gz
Note: The preceding log excerpts are only examples. Date, time, versions and environmental variables may vary depending on your environment.


Environment

VMware NSX-T Data Center
VMware NSX

Cause

  • After you successfully uploaded the MUB file, pre-checks were initiated.
  • Then the Upgrade Coordinator was upgraded, while the pre-checks are still ongoing.
  • Pre-check shouldn't be initiated until the Upgrade Coordinator is upgraded.
  • Due to incorrect procedure, pre checks triggered before UC is upgraded, a file is created in this location: /image/upgrade-coordinator/migration-progress.json

Resolution

In VMware NSX, Pre-check shouldn't be initiated until the Upgrade Coordinator is upgraded.

Workaround:

To workaround this issue, please proceed with the below steps on the orchestrator NSX Manager node:
  • Delete the file that was created due to the pre-check being run after the MUB file was upload: 
rm /image/upgrade-coordinator/migration-progress.json
  • Restart the upgrade coordinator to clean up stale mub-upload-in-progress flag file:
service upgrade-coordinator restart
  • Then retry UC upgrade by clicking on "Begin Upgrade" button on Upgrade UI, and proceed with upgrade.