Upgrade of Aria Operations 8.18.6 or 8.18.7 to VCF Operations 9.0.x is Unsupported
search cancel

Upgrade of Aria Operations 8.18.6 or 8.18.7 to VCF Operations 9.0.x is Unsupported

book

Article ID: 438994

calendar_today

Updated On:

Products

VCF Operations VCF Operations/Automation (formerly VMware Aria Suite) VMware Aria Operations (formerly vRealize Operations) 8.x VMware Cloud Foundation

Issue/Introduction

You may be asking:

"Can you upgrade Aria Operations 8.18.6 to VCF Operations 9.0.x?"

or

"Can you upgrade Aria Operations 8.18.7 to VCF Operations 9.0.x?"

WARNING: Upgrading from Aria Operations 8.18.6 or 8.18.7 to VCF Operations 9.0.x (including versions 9.0.0, 9.0.1, and 9.0.2) is explicitly unsupported. Attempting this upgrade can put the application in an unrecoverable state and may cause total data loss. If you find your environment in this failed state without an offline snapshot or standard backup beforehand, leave the system exactly as is and Contact Broadcom support . Self-troubleshooting without a backup can cause further permanent damage. Do not attempt service restarts, cluster online/offline operations, manual command-line changes, force reinstalls, or reboots.


This limitation applies whether you are using VMware Aria Suite Lifecycle or performing a direct upgrade via the Aria Operations admin UI. 

See KB: Unable to upgrade Aria Operations 8.18.6 to VCF Operations 9.0, 9.0.1 or 9.0.2 from Aria Suite Lifecycle Manager.

Attempting this upgrade via a manual .pak file update bypasses pre-flight checks and results in a database schema mismatch, leading to a catastrophic split-brain cluster failure.

Symptoms of a failed upgrade attempt:

  • The primary node may report version 9.0.2, while the data node reports version 8.18.6 or 8.18.7.
  • The upgrade halts at Step 7 (Applying operating system update).
  • The installation is not completing and admin UI keeps showing the installation in progress.
  • You observe the following error in the vcopsPakManager.root.apply_os_update.log file:

  • [vcopsPakManager - vcopsPakManager - HandleError - 630] - Exiting with exit code: 1, message: The action: apply_os_update on the pak file: "vRealizeOperationsManagerEnterprise-902025137843" previously failed, cleanup the system and remove_history before attempting again--exiting
  • The /pakManager/vRealizeOperationsManagerEnterprise-902025137843/apply_os_update_stderr.log shows:

    Error(1525) : rpm transaction failed

    ERROR - Script command: "/usr/bin/tdnf install -y $(/usr/bin/tdnf --repoid=local-pak-iso repoquery --available)" failed with exit code: 245

    ERROR 442986 [ops@4413 threadId="###############" threadName="MainThread"] [b2b - updateCoordinator - programExit] - Unable to upgrade packages.

     

  • The /storage/log/vcops/log/casa/casa.log shows:
    [com.vmware.vcops.casa.net.util.FailureRecoveryUtility.retry:66] - PakID=vRealizeOperationsManagerEnterprise-902025137843, slice=[IP_ADDRESS]: attempt failed operateOnSlice: org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 500: "{"error_message_key":"pak_manager.action_failed","error_arguments":["apply_os_update"],"error_message":""}"

Environment

  • VMware Aria Operations 8.18.6

  • VMware Aria Operations 8.18.7

  • VCF Operations 9.0.0

  • VCF Operations 9.0.1

  • VCF Operations 9.0.2

Cause

While both versions contain critical security updates, the Photon OS package manager identifies a package version conflict and regression risk during the upgrade from the 8.18.6 or 8.18.7 build base to 9.0.x. This conflict causes the package manager to intentionally abort the apply_os_update script. Because the OS update script halts mid-execution, the appliance operating systems are left in a corrupted, inconsistent state.

Resolution

DO NOT stage or execute any VCF Operations 9.0.x .pak file on an Aria Operations 8.18.6 or 8.18.7 cluster using the Aria Operations admin UI or Aria Suite Lifecycle. There is no Pre-Upgrade Assessment Tool (APUAT) or validation checks within the PAK file natively available to warn you via the appliance UI before attempting a manual .pak upgrade.

To proceed with a supported method, choose one of the following alternatives:

  1. Maintain the existing Aria Operations 8.18.6 or 8.18.7 cluster
    • Revert to snapshot or backup
    • Await a subsequent future release of VCF Operations 9.0.x update that explicitly adds version 8.18.6 or 8.18.7 to the approved source compatibility matrix.

  2. Deploy a new, greenfield VCF Operations 9.0.2 cluster via Fleet Management. Export your custom content (such as Dashboards, Super Metrics, Views, and Reports) from the 8.18.6 or 8.18.7 cluster and import them into the new 9.0.2 cluster.
Note: Historical time-series metric data cannot be migrated using the greenfield deployment method.


If you need assistance from Broadcom Support, manually capture logs from all cluster nodes from command line by following Collecting diagnostic information from VMware Aria Operations

Additional Information

 Please subscribe to this article to be notified of updates to this article.

Powered by Wolken Software