Cloud proxies upgrade failed in Aria Operations

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

The Aria Operations cluster has been recently upgraded.

In the Aria Operations Admin interface, Cloud Proxies menu:

• The IP address column shows as empty.
• Health status shows Outdated life status.

Environment

• Aria Operations 8.x

Cause

• Sometimes the Cloud Proxy booting process might get stuck due to an ongoing file system check on /dev/data/db. This can be verified using the journalctl command and the following events can be seen:
  • Finished File System Check on /dev/data/db.
  • Received SIGINT.
  • Failed to enqueue replace-irreversibly job for ctrl-alt-del.target: Unit ctrl-alt-del.target is masked.

• This /dev/data/db check, might encounter a hang or a prolonged state that prevents the boot from proceeding.

Resolution

1. The upgrade of the cloud proxies will start after the cluster has upgraded. Allow some time following the upgrade for the sequential upgrade of the cloud proxies to complete.

Note: The Ctrl+Alt+Delete can be used from vCenter > Cloud Proxy VM > Summary > Launch Web Console, as this action forces the system to bypass the stalled file system check (by sending an Interrupt Signal (SIGINT)), allowing the boot sequence to proceed.

2. If the above does not work, redeploy a new Cloud Proxy with the new version.

Additional Information

Aria Operations Two-Step Upgrade Flow