VCF Domain Upgrade - The upgrade fails at "NSX UPGRADE STAGE UPGRADE EDGE" stage in NSX-T EDGE Cluster

Products

• VMware SDDC Manager

Issue/Introduction

Error Details:

Stage: NSX_UPGRADE_STAGE_EDGE_POSTCHECK
Error Code: com.vmware.vcf.error.runtime.nsxt.edge.cluster.postcheck.failed
Description: ServiceUnavailable: An existing request to execute upgrade checks is in progress. Please wait and retry.

 

The following stages were completed in sequence:
NSX_UPGRADE_STAGE_CLUSTER_PRECHECK
NSX_PERFORM_BACKUP
NSX_UPGRADE_STAGE_SET_UPGRADE_PAYLOAD
NSX_UPGRADE_ACCEPT_UPGRADE_EULA
NSX_UPGRADE_STAGE_UPDATE_UC

When accessing the NSX-T UI on the Upgrade page we can see that the "Start Upgrade" button is greyed out. 
There is no progress happening on NSX-T, the SDDC Manager upgrade time-outs after a long delay.

The process only completes the NSX-T upgrade prechecks.

Environment

VCF 4.5.x

VCF 5.x

NSX-T 3.x

NSX-T 4.x

Cause

The issue is likely caused by the upgrade bundle issue.

Resolution

Remove the existing bundle using bundle cleanup tool, download the bundle again from the depot:

How to: Bundle Cleanup Utility

Once that is done you can start a new upgrade using the new bundle.