Error: 500 Internal Server Error [no body] - NSX Global Manager upgrade fails with error executing upgrade step 'null'

Products

• VMware NSX

Issue/Introduction

Symptoms:

During an NSX Global Manager upgrade (typically to version 4.2.x), the upgrade fails on the first node of a standby cluster.
The NSX Manager UI may show a 503 service unavailable or a spinning wheel.
The HTTP service on the affected node is down.
Running get upgrade progress-status from the admin CLI returns:

• % Error executing upgrade step 'null'

•  The upgrade-coordinator.log contains errors similar to:

• com.vmware.nsx.management.upgrade.rpcframework.UcRestRpcException: org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 Internal Server Error: [no body]

• Upgrade pre-checks fail with repository certificate errors and version sync status mismatches.

 

Environment

 VMware NSX

NSX Federation

 

Cause

The upgrade failure occurs due to a corrupted upgrade plan metadata state, often triggered by service restarts or communication failures during the Management Plane (MP) upgrade phase. This leaves the node in a partial upgrade state where binaries are updated but the database and service configurations are mismatched.

Resolution

This issue is currently under investigation. 

Workaround steps: 

1. Redeploy the Degraded Node following steps in Replace a manually deployed NSX Manager node when cluster is degraded
2. Perform a Rolling Reboot:
   • Perform a controlled, rolling reboot of all manager nodes in the cluster. This aligns the Upgrade Coordinator states across the Management Plane.
3. Resume Upgrade:
   • Re-run the upgrade pre-checks. Once they pass, resume the upgrade from the NSX UI.