"Import VCF Operations in Fleet Lifecycle" stage of "Deploy VCF Management Components" workflow fails or completes with warnings

Products

• VCF Operations

Issue/Introduction

This issue occurs during a VCF 9.0 to 9.1 upgrade with the following warnings:

• Failed to discover component of type 'VCFA'
• Failed to discover component of type 'VIDB'
• Failed to discover component of type 'OPS_LOGS'
• An unexpected error occurred in step discover_vrni_component_task
• Failed to add existing VCF Operations Networks
• Failed to add existing Log management

If your SDDC Manager, Fleet lifecycle, SDDC lifecycle, and VCF Operations are all at version 9.1.0.0300 or newer, review this section.

After "Deploy VCF Management Component" workflow completes, if one of the components in your VCF 9.0 deployment (such as VCF Automation) could not be add to Fleet lifecycle, you will see the following information.

1. VCF Operations UI will display banner to prompt users to Go to VCF Management tab, as some components are blocked from being upgraded
   
   
2. VCF Operations UI will report an alert for user attention
   
   
   
   (Note: Cancelling this alert from the VCF Operations Alerts panel dismisses both the top warning banner (example 1 above) and the VCF Management warning panel (example 3 below)
   
   
3. VCF Operations Build → Lifecycle → VCF Management tab shows warning panel with list of components that failed to be added to Fleet lifecycle, each linking to the corresponding task detail
   
   
   
   
   
4. Task details link shows the cause of the failure and link to this KB article for remediation steps.

 

If your SDDC Manager, Fleet lifecycle, SDDC lifecycle, and VCF Operations running versions before 9.1.0.0300, review this section.

After "Deploy VCF Management Component" workflow completes, it will be in a Failed state as shown below.

1. Upgrade failure in the VCF Operations UI
   
   
   
   
2. Example of the failed Import task in the VCF Operations UI

Environment

VCF Operations 9.1

Cause

The import step fails for one or more of the following reasons:

• Invalid credentials
• Invalid certificate
• Missing vCenter host
• Invalid component IPs

All conditions produce the same symptom. The failed task cannot be retried directly until the invalid data is cleared from VCF Operations.

Resolution

To resolve failures during the "Deploy VCF Management Components" task in a VCF 9.0 to 9.1 upgrade, you must use the attached script to clear invalid metadata and import 9.0 components.

Prerequisites:

Access required: 

• SSH access to the SDDC Manager appliance

Credentials required:

Credentials	Default Username	Purpose
SDDC Manager admin password	[e-mail]	Retrieving component FQDNs
VCF Services Runtime password	[e-mail]	Acquiring Fleet lifecycle API JWT
VCF Operations admin password	admin	Discover existing 9.0 components
VCF Automation password	vmware-system-user	Option 2 import
VCF Operations for networks password	admin@local	Option 2 import
Identity Broker password	vmware-system-user	Option 2 import
Log management password	admin	Option 2 import

 

What the Script Does:

To resolve the issue, use the attached script. The script runs interactively and guides you through two options. Only run the option(s) that apply to your version as identified below: 

• Option 1 — Clear stale data and restart the deploy task
• Option 2 — Manually import missing 9.0 components

Only run the option(s) that apply to your version as identified below:

Symptom	Option
Deploy VCF Management Components task completed with warnings	Option 2 only
Deploy VCF Management Components task failed	Option 1, then Option 2

Note: A timestamped log file is written automatically to:

/var/log/vmware/vcf/vcf_remediation_backups/remediation_YYYYMMDD_HHMMSS.log

Option 1 — Clear stale data and restart the deploy task

Run this first when the "Deploy VCF Management Components" task is in a FAILED state.

• Prompts for credentials once at the start.
• Backs up the existing legacy component data before making any changes.
• Deletes the invalid legacy LCM data from VCF Operations and waits until the deletion is confirmed.
• Triggers a retry of the "Deploy VCF Management Components" task. If the automatic retry does not succeed, the script prints manual instructions.

Option 2 — Manually import missing 9.0 components

Run this if Option 1 is complete (or not needed) and if any 9.0 components are still missing from Fleet.

• Shows a status table of which components are already added and which are still missing.
  
  
• Lets you select which components to add: VCF Automation, VCF Operations for networks, Identity Broker, and/or Log management.
  
  
• Runs pre-checks before adding each component and stops if any check fails:
  
  
  • Verifies the component is reachable over the network.
  • For VCF Operations for Networks: checks that the TLS certificate includes a valid SAN entry. If this check fails, the script directs you to KB 440459 to rotate the certificate before proceeding.
  • Validates credentials for all components. If the check fails for VCF Automation or Identity broker due to an expired password, the script directs you to KB 424323 to reset the vmware-system-user password before proceeding.
    
    
• Submits the add component request to Fleet lifecycle for each component that passes all checks.

Remediation Steps:

Note: All steps in this KB must be performed from the SDDC Manager appliance as root.

1. Download the remediate_deploy_mgmt_components.py script attached to this KB.
   
   
2. Copy the script to the /home/vcf/ directory on the SDDC Manager appliance:
   
   
3. SSH into the SDDC manager appliance with vcf user and then elevate to root with su.
   
   
4. Run the script 
   
   python /home/vcf/remediate_deploy_mgmt_components.py
   
   (The script writes a log file to: /var/log/vmware/vcf/vcf_remediation_backups/remediation_YYYYMMDD_HHMMSS.log)
   
   
5. Follow the prompts of the script to identify and remediate any detected issues.
   
   See What the Script Does above for a full description of each option's behavior.