NSX upgrade fails with version null error or resets at 70% during repository synchronization
search cancel

NSX upgrade fails with version null error or resets at 70% during repository synchronization

book

Article ID: 448394

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

During a VMware NSX upgrade (e.g., from 4.2.3.x to 4.2.4), the upgrade process exhibits one of the following behaviors:

  • The upgrade UI silently resets to the upload prompt after reaching 70%.
  • Importing a Pre-Check Upgrade Bundle (PUB) fails with error: Unable to start upgrade-coordinator upgrade. The version null is not recognized by the Manager.
  • The NSX Manager logs (/var/log/proton/nsxapi.log) contain entries: errorCode="MP30040" level="ERROR" subcomp="upgrade-coordinator" indicating Repository Sync status is not success.

Environment

  • VMware NSX 4.x

Cause

This issue occurs when the Upgrade Coordinator (UC) enters an indeterminate state due to a discrepancy between the management database and the physical files in the repository. Common triggers include:

  • Manual deletion of files within /repository/ to resolve disk exhaustion.
  • High disk usage on a specific manager node preventing complete synchronization.
  • Interrupted repository sync where the database marks a version as 'Present' but critical UC components are missing from the file system.

Resolution

Follow these steps to restore the repository and proceed with the upgrade:

  1. Restore the physical repository bits on the Orchestrator node by following the extraction steps in KB 322436.
  2. Trigger a repository sync on all non-orchestrator nodes via API:
  3. Log in to the NSX UI and navigate to the Upgrade tab.
  4. Instead of re-uploading the Precheck Upgrade Bundle (PUB), upload the Main Upgrade Bundle (MUB) for the target version.
    • Note: The MUB upload performs a clean re-extraction that does not depend on existing repository bits, effectively un-wedging the Upgrade Coordinator.
  5. Upgrade the Upgrade Coordinator through the UI. The UC will automatically handle the extraction and synchronization across the cluster.
  6. Once the UC upgrade completes, click Previous in the UI to run pre-checks and proceed with the upgrade.

Additional Information

Note: In NSX 9.1 and above, a prohibitive check prevents the re-upload of a PUB file of the same version specifically to prevent users from encountering this indeterminate database desync state.

Powered by Wolken Software