Aria Lifecycle Manager 8.16 Upgrade Issue: "Product Successfully Upgraded" Without Performing the Upgrade

Products

• VMware Aria Suite

Issue/Introduction

When attempting to upgrade Aria Lifecycle Manager from version 8.14.0.4 to version 8.16, the process indicates that the upgrade was successfully completed without actually performing the upgrade. This occurs whether using the "Check Online" method or by mounting the upgrade ISO directly to the VM. All pre-checks pass, but the upgrade immediately displays "VMware Aria Suite Lifecycle successfully upgraded," even though the upgrade has not been executed.

From the logs: 

/var/log/vmware/capengine/core-engine.log

workflow_handler.go:154: Error in executing request. Error: Operation not allowed in the current state of workflow
engine.go:127: Failed to execute workflow request. Error: Operation not allowed in the current state of workflow

/var/log/vrlcm/vmware_vrlcm.log

INFO  [http-nio-8080-exec-10] c.v.v.l.u.u.LCMUpgradeUtil -  -- {"id":"ms_error_forbidden","locale_id":"en-us","default_message":"Forbidden: Operation not allowed in the current state of workflow","args":["Operation not allowed in the current state of workflow"]}

Environment

VMware Aria Suite Lifecycle 8.x

Cause

The upgrade failure is caused by the workflow engine (cap-workflow-engine) being in an incorrect or inconsistent state. The error logs indicate that an operation is "not allowed in the current state of workflow," which suggests that the workflow.db file, which maintains the state of the workflow, has become corrupted or misaligned. This prevents the cap-workflow-engine service from executing the upgrade process correctly, leading to the false success message.

Resolution

To resolve this issue, follow these steps:

• Stop the cap-workflow-engine service:
  
  systemctl stop cap-workflow-engine.service
  
  
• Remove the corrupted workflow database:
  
  rm -rf /storage/lifecycle/capengine/workflow.db
• Restart the cap-workflow-engine service:
  
  systemctl start cap-workflow-engine.service
  
  

This process will recreate a new workflow.db file in the same location, resetting the workflow state.
Re-trigger the upgrade process using either the "Check Online" method or by remounting the ISO to perform the upgrade again.

Additional Information

 

Ensure all necessary backups are completed before performing the above steps.
Monitor the upgrade logs (/var/log/vmware/capengine/core-engine.log and /var/log/vrlcm/vmware_vrlcm.log) to verify that the upgrade progresses without errors after applying the fix.