Aria Operations upgrade to 8.18.7 stuck on stage 8 in disconnected environments
search cancel

Aria Operations upgrade to 8.18.7 stuck on stage 8 in disconnected environments

book

Article ID: 444860

calendar_today

Updated On:

Products

VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

  • When attempting to upgrade VMware Aria Operations from 8.18.6 to 8.18.7, the process may hang or become stuck for several hours at the following stages: "8 of 14 - Certificate renewed" or "7 of 14 -Preapply Validated"



  • The following errors can be seen in the respective log files: 

    var/log/vmware/vcops/vcops_upgrade_<timestamp>.log 
    YYYY-MM-DDTHH:MM:SS DEBUG - b2b -updateCoordinator - runScript - stderr: "warning: /storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-818725423537/extracted/salt-3006.17-0.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 73d76b3f: NOKEY 
    ls: cannot access '/run/salt/minion': No such file or directory 
    ls: cannot access '/run/salt/minion': No such file or directory 
    chown: invalid user: ‘%{_MN_LCUR_USER}:%{_MN_LCUR_GROUP}’ 
    warning: %posttrans(salt-minion-3006.17-0.x86_64) scriptlet failed, exit status 1 " 
    YYYY-MM-DDTHH:MM:SS DEBUG - b2b -updateCoordinator - runScript - exit code: 0 
    YYYY-MM-DDTHH:MM:SS DEBUG - b2b *updateCoordinator - rpm_group_install - Completed Upgrade RPM(s): "/storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-818725423537/extracted/salt*.rpm" 
    YYYY-MM-DDTHH:MM:SS INFO - b2b -updateCoordinator - os_upgrade - Restoring Salt-Minion configuration 
    YYYY-MM-DDTHH:MM:SS INFO - b2b -updateCoordinator - restore_salt_config - Backup directory '/etc/salt/minion_backup/' does not exist. Nothing to restore. 
    YYYY-MM-DDTHH:MM:SS DEBUG - b2b -updateCoordinator - remove_packages - Removing packages 
    YYYY-MM-DDTHH:MM:SS8 DEBUG - b2b -updateCoordinator - runScript - Script command: "/usr/bin/tdnf list installed | grep photon_vasecurity"
  • /storage/vcops/logs/pakManager/vcopsPakManager.root.apply_os_update.log:  
    DEBUG - Script command: "/usr/bin/tdnf list installed | grep photon_vasecurity"
    DEBUG - stdout: ""
    DEBUG - stdout: ""
    2026-06-16T23:41:55 DEBUG - stdout: ""
    DEBUG - stderr: "Error(1229) : Timeout was reached
    Error: Failed to synchronize cache for repo 'VMware Photon Linux 5.0 (x86_64) Snapshot for Updates'
    

Environment

Aria Operations 8.18.7

Cause

During the upgrade to Aria Operations 8.18.7, a required RPM package attempts to download from an online repository. If internet access is unavailable (such as in disconnected Aria Operations environments), the upgrade cannot resolve this dependency and gets stuck.

Resolution

To bypass this situation and complete the upgrade currently, follow the below workaround:

  1. Roll back the Aria Operations appliance(s) to the snapshot taken prior to initiating the upgrade.

  2. Temporarily enable internet access for the appliance to https://packages.broadcom.com port 443.

  3. Re-attempt the upgrade to 8.18.7.

If internet access cannot be enabled in disconnected Aria Operations environments please follow the alternative workaround instructions below:
 
 
Alternative Workaround:
 
The following steps need to be performed on all available nodes:
  1. SSH to the primary node using root.
  2. Look for the stuck package manager process: ps aux | grep tdnf
    1. The output should show as below:
  3. Terminate the stuck process: kill -9 ####(terminate only "tdnf list installed")
  4. As soon the process is killed, the node should reboot in about 20-30 seconds.
  5. If not, repeat step-2 and 3.
  6. Monitor the upgrade progress in the UI. The installer skips the online check and proceeds to the next local stage.
  7. Wait for the appliance to reboot automatically to complete the upgrade.
  8. Validate the upgrade status in the 'Software Update'.
  9. If the remaining nodes are in stuck state, repeat steps from 2-5 on those nodes.

This issue will be resolved in a future releases of Aria Operations.

Additional Information

Subscribe to this KB or refer the release notes for forthcoming product releases for any updates.
Reference: How to Subscribe to the KB