Upgrade to VMware Cloud Foundation 9.1 stalls at import legacy components stage

Products

• VCF Automation

Issue/Introduction

• During a VMware Cloud Foundation (VCF) Operations upgrade from version 9.0 to 9.1, the process stalls at the "Install VCF Operations - Import legacy components using SDDC lifecycle" stage.
• As a symptom of the upgrade breaking, you may also encounter issues where you attempt to bypass the error by creating an organization via the Quick Start menu. This can result in an organization being created with incorrect characters in the display name that you are subsequently unable to edit, disable, or remove.
• Post-upgrade, the import of VCF management services fails to install the Salt RaaS, Salt Master, and Telemetry components. These components are required before VMware Aria Automation can be successfully upgraded.

Environment

 

• VMware Cloud Foundation (VCF) Operations 9.0 to 9.1

• VCF Automation 9.0

 

Cause

• The Fleet lifecycle attempts to import legacy components (the 9.0.x inventory data) from VCF Operations but fails due to missing infrastructure properties (NULL values).
• This typically occurs if the VCF Automation component was previously removed and re-added to the fleet management node's inventory.
• Re-importing brings in the component properties but fails to import infrastructure properties (such as DNS and NTP details). Because these values are NULL, the import of VCF Operations component details fails. Retrying the failed task is unsuccessful as it reuses the same payload.

Resolution

To resolve this issue, manually deploy the pending components using the Fleet lifecycle API, and then import the remaining components via the VCF Operations UI.

Step 1: Identify Missing Components

1. Navigate to Build > Lifecycle > VCF Management > Components in the VCF Operations UI.

2. Compare the deployed components with those present in version 9.0.x. If they are listed, no further action is needed. If they are missing, proceed to Step 2.

Step 2: Deploy Pending Components via API

Use the Fleet lifecycle API to deploy the missing components (Salt RaaS, Salt Master, Telemetry, and License Server).

1. Download and install an API client (such as Bruno or Postman).

2. Gather the necessary Fully Qualified Domain Names (FQDNs) and credentials:

   • VCF Services runtime FQDN

   • Fleet components FQDN

   • Username ([email protected])

   • Password (typically your vmware-system-user password)

3. Configure the following variables in your API client environment:

   • vcf_services_runtime_fqdn

   • fleet_component_fqdn

   • vcf_sr_username

   • vcf_sr_password

   • sddc_lcm_id

4. Fetch the Authentication Token by executing a POST request:

   • API Endpoint: https://{{vcf_services_runtime_fqdn}}/api/v1/identity/token

   • Headers: Content-type: application/x-www-form-urlencoded

   • Body:

      

     grant_type: password
     username: {{vcf_sr_username}}
     password: {{vcf_sr_password}}
     

5. Fetch the SDDC LCM ID using the API and save it to your environment variables (the Fleet lifecycle calls the SDDC lifecycle to execute the installation task).

6. Execute the respective API calls to install Salt RaaS, Salt Master, and Telemetry.

7. Once the components are installed, clear the legacy VMware Aria Automation 9.0 environment from vSphere.

Step 3: Import VMware Aria Automation

1. In VCF Operations, navigate to Build > Lifecycle > VCF Management > Components.

2. Click Add Component.

3. Select VCF Automation.

4. Select your VCF Instance.

5. Enter the VCF Automation Primary VIP.

6. Enter the vmware-system-user password to import VMware Aria Automation into the Fleet lifecycle inventory so it is available for the 9.1 upgrade.

Step 4: Import VCF Operations for Networks

1. In VCF Operations, navigate to Build > Lifecycle > VCF Management > Components.

2. Click Add Component.

3. Select VCF Operations for Networks.

4. Enter the Platform Node's IP address.

5. Enter the admin password (admin@local).