Error: Undefined upgrade task in VCF 9.1 Operations lifecycle interface

Products

• VMware SDDC Manager / VCF Installer
• VCF Operations

Issue/Introduction

Stale upgrade plan components appear in the VCF Operations lifecycle 9.1 interface. as Log Management lifecycle interface after a failed VCF 9.1 installation or upgrade cleanup.

• Components typically show a status:
  "Ready for upgrade Prechecks failed" and target versions such as 9.1.0.0.########. Attempting to re-run prechecks or resume the upgrade results in a VCF_LCM_FLEET_UPGRADE_400_INVALID_UPGRADE_PLAN_SPEC error.
• After running the cleanup_component script attached to Failed component cleanup during VCF 9.1 installation or upgrade, an "Internal Server Error" related to binaries may be present:
  Unable to retrieve information about required binaries. Internal Server Error for service fleet-upgrade-service. Verify logs for service fleet-upgrade-service.
  
• An undefined/failed upgrade entry remains visible as well after successful VCF Automation upgrade.
  
• Environmental assessment confirmed no active concurrent upgrade processes or precheck tasks.

Environment

VCF Operations 9.1
VCF Automation 9.1

Cause

A stale upgrade_plan_component ID remains in the Fleet Lifecycle database (vcffleetlcmdb) after the cleanup_component.py script is executed. The UI renders these orphaned records as undefined tasks because the primary inventory references have been removed, but the task metadata persists.

Resolution

1. Login into VCF Operations and go to  Lifecycle > Build > VCF Management > Component > VCF services runtime 

   

2. Identify a Control Plane node IP:
   
   

3. Copy the attached script to the SDDC Manager /home/vcf/ directory.

4. SSH into the SDDC Manager with the vcf user and then elevate to root with su.
   
   
5. Run the script 
   
   python cleanup_upgrade_plan_component.py

Provide the VSP services runtime Control Plane node IP from step 2 when prompted.
Provide the vmware-system-user password when prompted. 
Follow the script instructions to identify and cleanup the stale upgrade_plan_component id.
   
   
6. Refresh the VCF Operations UI to confirm the failed upgrade task has been removed.

Additional Information

Failed component cleanup during VCF 9.1 installation or upgrade