VCF Automation Upgrade to 9.1 fails at Power Off Source Cluster step with Error vmsp_upgrade_vcfa failed

Products

• VCF Automation

Issue/Introduction

VCF Automation upgrade fails during the component upgrade stage after successfully completing the upgrade pre-checks. You see the following error messages during the failure:

• Info : Power Off Source Cluster. Status: Failed

• Error : Step vmsp_upgrade_vcfa failed.

 

Environment

VCF Automation 9.0 / 9.1

VMware Cloud Foundation (VCF) 9.1

Cause

This failure occurs due to a discrepancy in the Datastore path. If the datastore location of the VCFA VMs was changed in vCenter (e.g., via Storage vMotion), the VCFA cluster configuration does not automatically update to reflect this new datastore path. As a result, the VM datastore information stored in the VCFA configuration differs from the actual information vCenter holds, causing the cluster power-off task to fail.

 

Resolution

To resolve this issue, you must verify the path discrepancy, roll back the failed upgrade, correct the datastore location, and retry the upgrade.

1. Check the datastore path in the VCFA 9.0 configuration: Log in to the VCFA appliance using SSH as root and run the following command to retrieve the configured datastore path:

   k get pd vmsp-platform -n vmsp-platform -o yaml
   
   spec:
     values:
       provider:
         vsphere:
           datastore:
   

2. Check the actual datastore path in vSphere Client: Log in to the vCenter Server using the vSphere Client and verify the current datastore location of the VCFA VMs. (Note: The configuration from Step 1 must exactly match the path seen in vCenter.)

   

3. Rollback the environment: If the datastore paths do not match, you must roll back the environment from VCFA 9.1 to 9.0. Note: The rollback process is not supported as a standalone user task. You must create a Support Request to engage Broadcom Technical Support for assistance with the rollback.

4. Once the rollback to 9.0 is successfully completed, restore the datastore to its original storage folder path (Drag And Drop).

5. Retry the Upgrade: After the VMs have been moved to the correct datastore path, retry the VCF Automation upgrade.