SDDC manager UI reports "New update cannot be started while another ESX Server update is in progress or scheduled for this domain" when attempting to cancel/retry the upgrade task for a single cluster

Products

• VMware vSphere ESXi

Issue/Introduction

• When an ESXi cluster upgrade fails on the SDDC manager it is necessary to fix the underlying issue and then re-trigger the upgrade of the failed cluster from the SDDC manager. 
• If the failed cluster was part of a parallel upgrades, then the users will encounter an issue, where the current system design doesn't allow for independently re-triggering the upgrade for the failed cluster.
• The message "New update cannot be started while another ESX Server update  is in progress or scheduled for this domain" will be seen under the Updates tab on the "Workload domain"

• When this occurs the following two options can be performed. 
• Cancel upgrade: The cancel upgrade, cancels upgrade task for all the cluster triggered as part of parallel upgrade process, this extends the downtime for all the clusters. 
• Mandatory Waiting Period: Wait until the ongoing upgrade completes on the other clusters and then retry the upgrade on failed clusters. 

 

Environment

VCF 5.x 

Cause

• This behavior was introduced in VCF 5.x, differing from VCF 4.x, to prevent accidental cluster upgrades and avoid unexpected downtime.
• This UI restriction ensures upgrades are only started intentionally.
• Another key observation is how these upgrades are tracked. All cluster upgrades started at the same time via the parallel option are managed under just one task ID

Resolution

While the UI limits parallel upgrades, individual clusters can still be upgraded concurrently using the v1/upgrade API. This API allows multiple cluster upgrades to be performed independently, each with its own task ID for granular control.

• For detailed instructions and examples, refer to the "Cluster upgrade API" section in the official documentation: Upgrades APIs
• The API needs to be invoked with a separate payload for each cluster upgrade in parallel. To do the same. 
  • Fetch the cluster ID by following the techdocs: Retrieve the Workload Domain vSphere Cluster ID from SDDC Manager  
  • Navigate to Developer Center -> API explorer on the SDDC manager 
  • In the search box, search for: v1/upgrade API 
  • Expand POST
  • Create a payload in the following manner and click on execute .  

{

  "bundleId": "0c69c30d-5edd-4c08-812d-7d793ef5fd43",
  "parallelUpgrade": true,
  "resourceType": "CLUSTER",
  "draftMode": false,
  "resourceUpgradeSpecs": [
    {
      "resourceId": "7#######-1####-4###-8###-c###########",
      "upgradeNow": true,
      "type": "CLUSTER",
      "personalitySpec": {
        "personalityId": "9#######-8####-4###-8###-2###########"
      },
      "esxUpgradeOptionsSpec": {
        "enableQuickboot": true,
        "evacuateOfflineVms": true,
        "enforceHclValidation": false,
        "enableQuickPatch": false,
        "disableDpm": true,
        "disableHac": false,
        "preRemediationPowerAction": "DO_NOT_CHANGE_VMS_POWER_STATE",
        "esxUpgradeFailureAction": {
          "action": "RETRY",
          "retryCount": 3,
          "retryDelay": 300
        }
      }
    }
  ]
}

• For fetching the cluster ID, follow the Techdocs.