NSX Manager Upgrade Failure at "Step 7 (Product Version Update)" with error proxying request "error_code": 36580

Products

• VMware NSX

Issue/Introduction

During an NSX upgrade, the Management Plane upgrade may encounter a failure at Step 7 (Product Version Update) with below error:

/var/log/upgrade-coordinator/upgrade-cordinator.log.
[YYYY:MM:DD:HH:mm:ss] INFO http-nio-127.0.0.1-7442-exec-5 UpgradeCoordinatorFacadeImpl 74936 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Component: MP, status: FAILED, % complete: 95.0, details: [MPP] Upgrade failed at mp_update_version_uu_name with error : Update Version failed for node cd9f3442-#########-c318eb954380 with error [UC] Error in rest call. url= /api/v1/cluster/cd9f3442-#########-c318eb954380/node/update-product-version , method= POST , response= {"error_code": 36580, "error_message": "Error proxying request to: cd9f3442-#########-c318eb954380.", "module_name": "node-services"} , error= 500 Internal Server Error: "{"error_code": 36580, "error_message": "Error proxying request to: cd9f3442-#########-c318eb954380.", "module_name": "node-services"}" ..., canSkip: false

Environment

VMware NSX

Cause

The failure is attributed to a transient service timeout. The Upgrade Coordinator is unable to establish a connection with the local node-services API on the affected node.

Resolution

To resolve this issue, perform a manual retry of the upgrade process from the NSX Manager UI by clicking the Retry button to retrigger the API version update workflow.