NSX upgrade via SDDC fails at the "SET UPGRADE PAYLOAD" step

Products

• VMware NSX
• VMware SDDC Manager / VCF Installer

Issue/Introduction

• The NSX component upgrade is failing from the SDDC Manager at the "Set Upgrade Payload" step with an error similar to:

  NSX Upgrade Coordinator - NSX Payload verification failed

• The SDDC displaying correct NSX target, but the Upgrade Co-Ordinator (UC) on the NSX UI is showing an old upgrade information.

• The workaround in "NSX Payload verification failed" Error during NSX upgrade via SDDC does not resolve the issue.

Environment

VMware NSX
VMware SDDC Manager

Cause

The SDDC failed copy the necessary files to upgrade the NSX Upgrade Coordinator

Resolution

To work around the issue:

• Option #1: 
  1. Upgrade the UC manually by uploading the PUB file to the UC in the NSX UI. 
  2. Resume the upgrade in the SDDC

• Option #2: Follow the NSX Out of Band (OOB) upgrade approach, by completing the NSX upgrade from the UC in the NSX UI and then sync the SDDC status:
  
  
  • For SDDC Manager 5.2 or higher:
    • Follow the steps in KB,  Synchronize inventory versions after out-of-band upgrade in a VMware Cloud Foundation Environment
      
      
  • For SDDC Manager 5.1 or less:

1. 1. 1. Login to SDDC manager VM via SSH and sudo to root account.
      2. Get VCF deployed NSX Cluster Inventory Ids associated with domains with the following api. Note down the "id" (the id of the NSX cluster) and "version" (current version) fields for each NSX.

         root@sddc-manager [ ~ ]# curl localhost/inventory/nsxt| json_pp

      3. For each NSX id, update the version to the correct version with the following API:

         curl localhost/inventory/entities/<<NSX ID>> -X PATCH -d '{"type":"NSXT_CLUSTER","status":"ACTIVE","version":"<<VERSION>>"}' -H 'Content-Type:application/json'