vCenter 9.0 to 9.1 RDU upgrade fails at VCENTER_UPGRADE_PREPARATION due to certificate download failure from target vCenter appliance
search cancel

vCenter 9.0 to 9.1 RDU upgrade fails at VCENTER_UPGRADE_PREPARATION due to certificate download failure from target vCenter appliance

book

Article ID: 442134

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • vCenter Reduced Downtime Upgrade (RDU) from version 9.0 to 9.1 fails during the VCENTER_UPGRADE_PREPARATION stage with the SDDC Manager UI displaying the following error:

    vCenter upgrade failed at the VCENTER_UPGRADE_PREPARATION stage. Remediation Message: Please retry the upgrade once the upgrade is available again. Contact GSS for assistance, if the retry fails. Note: Check if there are any errors related to vCenter upgrade rollback failure. If so, perform the remediation steps for the rollback failure before performing the upgrade remediation steps.

  • The SDDC Manager LCM log (located at /var/log/vmware/vcf/lcm/lcm-debug.log) reports a failure to download the target certificate:


    YYYY-MM-DDThh:mm:ss DEBUG [vcf_lcm,0000000000000000,0000,upgradeId=######-####-####-####-########,resourceType=VCENTER_PREPARE,resourceId=########-####-####-####-########,bundleElementId=######-####-####-####-########] [c.v.e.s.l.p.i.v.RduStatus,Upgrade-3] RDU status result: {"description":{"id":"com.vmware.vcenter.deployment.migration_upgrade.desc","defaultMessage":"Migration Upgrade of vCenter Server appliance.","args":[],"localized":"Migration Upgrade of vCenter Server appliance
    ....
    "status":"FAILED","startTime":"<YYYY-MM-DDThh:mm:ss>","endTime":"<YYYY-MM-DDThh:mm:ss>","progress":{"completed":##,"total":100,"message":{"id":"vcenter.deployment.migration_upgrade.stage.target_deployment.progress_message","defaultMessage":"Deploying the new version appliance. This operation can take some time.","args":[],"localized":"Deploying the new version appliance. This operation can take some time."}},"error":{"errorType":"ERROR","messages":[{"id":"vcenter.deployment.migration_upgrade.stage.cannot_download_target_cert_error","defaultMessage":"****","args":[],"localized":"Failed to download the public certificate from the deployed target vCenter Server Appliance"},{"id":"*****","defaultMessage":"Ensure that the target machine is reachable and the certificates on the machine are correctly formatted.","args":[],"localized":"*****"}]}}},"upgradeInfo":{"remainingReplicationData":####}}

  • The source vCenter Server VLCM logs (located at /var/log/vmware/vlcm/vlcm.log) contain corresponding connection failures:

    YYYY-MM-DDThh:mm:ss INFO vlcm 33213 [vc@4413] [locerrors/vapierrors.go:39] [opID=<opID>] Parsing vAPI error json to real object
    YYYY-MM-DDThh:mm:ss INFO vlcm 33213 [vc@4413] [locerrors/vapierrors.go:48] [opID=<opID>] Error type: map[Data:null ErrorType:ERROR Messages:[{"Id":"vcenter.deployment.migration_upgrade.stage.cannot_download_target_cert_error","DefaultMessage":"Failed to download the public certificate from the deployed target vCenter Server Appliance","Args":[],"Params":null,"Localized":"Failed to download the public certificate from the deployed target vCenter Server Appliance"},{"Id":"vcenter.deployment.migration_upgrade.stage.cannot_download_target_cert_resolution","DefaultMessage":"Ensure that the target machine is reachable and the certificates on the machine are correctly formatted.","Args":[],"Params":null,"Localized":"Ensure that the target machine is reachable and the certificates on the machine are correctly formatted."}]]

Environment

vCenter Server 9.X

Cause

An active proxy configuration enabled on the source vCenter Server intercepts and blocks communication to the newly deployed target vCenter Server appliance. This routing behavior prevents the source vCenter Server from successfully fetching the target vCenter Server's public certificate during the VCENTER_UPGRADE_PREPARATION stage.

Resolution

To resolve the issue and proceed with the Reduced Downtime Upgrade (RDU), implement one of the following methods to bypass the communication blockage, then retry the upgrade:

Method 1: Add the Target vCenter Server to the NO_PROXY List (Recommended)

  1. Log in to the source vCenter Server Appliance Management Interface (VAMI) via a web browser (https://<source_vcenter_fqdn>:5480).

  2. Navigate to the Networking section and select Proxy Settings.

  3. Edit the proxy settings and add the target vCenter Server's IP address or FQDN to the NO_PROXY exceptions list. This forces a direct connection and bypasses the external/forward proxy without requiring a restart.

  4. Verify connectivity by connecting via SSH to the source vCenter Server and running a curl or wget command against the target vCenter Server.

  5. Return to the SDDC Manager UI and retry the vCenter Server upgrade.

Method 2: Update the External Forward Proxy Rules

  1. Access your organizational external/forward proxy configuration.

  2. Ensure that the proxy rules explicitly allow traffic from the source vCenter Server to the target vCenter Server appliance (e.g., if rules currently only allow *.vmware.com or *.broadcom.com, temporarily allow traffic specifically to the problematic target server).

  3. Verify connectivity by connecting via SSH to the source vCenter Server and running a curl or wget command against the target vCenter Server.

  4. Return to the SDDC Manager UI and retry the vCenter Server upgrade.

Method 3: Temporarily Disable the Proxy Configuration Note: Use this method only if the vCenter Server does not require internet access during the upgrade.

  1. Log in to the source vCenter Server Appliance Management Interface (VAMI) via a web browser (https://<source_vcenter_fqdn>:5480).

  2. Navigate to the Networking section and select Proxy Settings.

  3. Temporarily disable the proxy configuration.

  4. Return to the SDDC Manager UI and retry the vCenter Server upgrade.

  5. Post-upgrade, log in to the newly upgraded target vCenter Server VAMI and re-enable the required proxy configurations.

Workaround:
If the above methods are not feasible due to organizational security policies, execute the standard In-place (Non-RDU) upgrade method via SDDC Manager as a workaround.