Pre-update system validation failed with error "...At least one Remote Collector found..."

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

1. Pre-update System Validation failed with the following warning as shown in the screenshot:

   "Checking Existence of Remote Collectors

   Validation Failed. A least one Remote Collector found

   Replace all Remote Collectors with generic Cloud Proxies"

 

2. When upgrading Aria Operations to 8.17.1 via Aria Suite Lifecycle (LCM), similar errors and details shown below are presented in LCM:

Error Code: LCMVROPSYSTEM25008

VMware Aria Operations upgrade failure

[{"rel":"pak_cluster_status","href":"https://vrops_IP_or_FQDN/casa/upgrade/cluster/pak/vRealizeOperationsManagerEnterprise-817123642532/status"},{"rel":"pak_current_activity","href":"https://vrops_IP_or_FQDN/casa/upgrade/cluster/pak/reserved/current_activity"}]

Environment

Aria Operations 8.14 and below

Aria Lifecycle Manager 8.16

Cause

In Aria Operations 8.14 and below environment, on admin UI > Cloud Proxies page, there are lightweight Cloud Proxies converted from Remote Collectors (RCs) as shown below:

RCs are no longer supported since vRealize Operations 8.10. See the Product Support Notice for more details.

The upgrade process from 8.10 through 8.14 converts RCs to lightweight Cloud Proxies (CPRCs), but RCs/CPRCs were completely deprecated in 8.16. Aria Operations 8.14.1 is the last version to support RCs. It's not supported to upgrade to 8.16 or beyond with legacy RCs or CPRCs in the cluster. See details Remote Collectors in Aria Operations 8.16 Release Notes.

Resolution

Before proceeding with the following steps, to save time during the process, make sure snapshots of Aria Operations nodes are taken by following How to take a Snapshot of VMware Aria Operations.

 

Adding new Collector Groups via LCM

1. Go to Settings > Binary Mapping and upload the "with_cp" upgrade PAK binary of Aria Operations.

2. On the left pane, click on Environment card, then click on VIEW DETAILS on the tile that's for the Aria Operations environment

3. Once on the product page, click  + ADD COMPONENTS next to the product name and version. See example below:

4. Follow the configuration wizard to fill out the information and add new Collector Group(s) for the new Cloud Proxies. You cannot mix Cloud Proxy types in the same Collector Groups. See details Remote Collectors in Aria Operations 8.16 Release Notes. 

 

As it's not possible to remove RCs through the LCM, we have to do this manually from within Aria Operations UI by following the steps below:

Removing the RCs in Aria Operations UI

1. In the Aria Operations UI, if more than one RC is in the environment, re-assign the adapter instance from the one being removed to the remaining RCs. If there is only one RC, follow the steps below regarding adding new CPs, once new CPs are added and the adapter instances are re-assigned to them, then remove the RC.

2. In the vCenter, take a snapshot with the memory of LCM.

3. If RCs' network settings (IP addresses, subnet masks, default gateway, etc) will be reused for new CPs, gather the information by locating the VMs in vCenter, then go to Configure > Settings > vApp Options. If vApp Options is not available, SSH into RCs as root and execute the following commands to gather the information.

Example:

Command for IP address and subnet mask:

ifconfig

Command for the gateway:

ip route | grep default 

Command for DNS servers:

systemd-resolve –status | grep “DNS Servers” for DNS servers

4. Once the above information is gathered, go to Aria Operations UI > Data Source > Cloud Proxies, click on the vertical ellipsis next to the RC that needs to be removed, and click Delete.

5. Go to Aria Operations > Administration > Collection Status, and confirm that all metrics are collected by the other RCs.

6. Also check the Adapter Instances to make sure all adapters are receiving.

7. Verify that the old RCs are no longer present in LCM under Environments. If not, run an inventory resync, which should remove the old RCs.

8. Go to vCenter and rename the old RCs that were removed in case their names will be reused for the new CPs. 

 

Adding CPs via LCM

1. Go to LCM UI, navigate to Environments> Operations > + Add Components.

2. Click on Trigger Inventory Sync.

3. Click Components > Add (+) > Cloud Proxy.

4. Fill in the configuration details of VM name, FQDN, IP Address, and Node Size.

5. Run the Precheck.

6. Go to Settings > Binary Mapping > Product Binaries, and make sure the Install Product Binaries are available for VMware Aria Operations. After the download, the OVA file for the Cloud Proxy is downloaded as well. (See example for Aria Operations 8.14.1)

7. Review the summary, recheck the network configurations, and start the deployment.

8. A request flow with be showing in LCM. In vSphere Client > Recent Tasks, you can see the OVF is deployed, and configured.

9. In Aria Operations UI, wait for the CP to be found. Then move the CP to the new Collector Group that was created in the beginning.

10. Run the above steps to add additional CPs.

11. Change the Collector Group for all accounts currently using the old RC Group into the new Collector Group.

12. Run an inventory sync for Aria Operations via LCM, all changes should be reflected.

13. In Aria Operations UI, go to Administration > Collection Status again and check the CP's health and usage. Check if all accounts/metrics are collected.

The final step of all, hard power off the former RCs' VMs in vSphere Client, and delete them from the environment.

After the above is done, proceed with upgrading Aria Operations from 8.14.1 and below to 8.16 and above via LCM.