After upgrading Aria Operations dashboards are no longer working for VMs

Products

VMware Aria Suite

Issue/Introduction

Built-in and custom dashboards for VM are blank and not populating.

Time and Capacity remaining shows ? Days.

Metrics for VMs are collecting and can see individual metrics.

In the analytic logs there are numerous errors similar to:

ERROR [Cache Loader worker thread 11] com.integrien.analytics.dataobject.ProblemAlertManager.init - Error creating alert definition managers in the Problem alert manager of resource : ######
java.lang.RuntimeException: AlertDefinitionManager creation: Evaluation plan can't be empty
at com.integrien.analytics.dataobject.AlertDefinitionManager.<init>(AlertDefinitionManager.java:31) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.dataobject.ProblemAlertManager.init(ProblemAlertManager.java:150) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.dataobject.ProblemAlertManager.<init>(ProblemAlertManager.java:114) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.dataobject.ResourceCache.initAlertManagers(ResourceCache.java:286) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.dataobject.ResourceCache.<init>(ResourceCache.java:326) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.vm.BadgeResourceCache.<init>(BadgeResourceCache.java:94) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.vm.VMResourceCache.<init>(VMResourceCache.java:81) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.alive.dbaccess.AnalyticsDBUtil.createResourceCache(AnalyticsDBUtil.java:471) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.alive.dbaccess.AnalyticsDBUtil.preloadResourceCache(AnalyticsDBUtil.java:550) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.alive.dbaccess.AnalyticsDBUtil$LoadResourceCacheWorkItem.run(AnalyticsDBUtil.java:123) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.alive.common.util.ThreadPool$WorkerItem.run(ThreadPool.java:275) ~[vrops-adapters-sdk.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.lang.Thread.run(Unknown Source) ~[?:?]

Environment

Aria Operations 8.17.1

Aria Operations 8.18

Cause

This is an issue with resource cache getting stuck due to bad alert definitions.

Resolution

The alert definitions that are triggering the issue will need to be removed.

1. 1. From the UI log in as admin.
  2. Navigate to Configure > Alerts > Alert Definitions
  3. On the upper right click on the Filter button.
  4. For Object Type use "Virtual Machine".
  5. Click on View More Filters and for Defined By type "user".
  6. Hit the Apply button to filter. This will list all the custom alert definitions for VMs.
  7. Click on each alert definition listed and look for a red circle with a line through it (not permitted symbol). These are the alerts definitions that will need to be removed.
  8. For each of the alert definition to be removed:
    1. Click on the 3 dotted menu and click export to backup the alert.
    2. Click on the 3 dotted menu and click delete to remove the alert.
    3. If you get a message saying the "Following Notifications are set" preventing you from deleting the alert you'll need to remove the notifications listed in the message.
  9. To delete the notification related to the alert definition:
    1. Navigate to Configure > Alerts > Notifications.
    2. Filter by name of each notification that need to be removed.
    3. Click the 3 dotted buttons to export if you want to back it up and then delete to remove.
    4. Do this for all the notifications set for the alert definition.
  10. Go back to Alert Definitions and try to remove the alert definition again.
  11. After removing the alert definitions wait for about 30 minutes.
  12. Check the dashboards and Time/Capacity remaining of any VM objects to verify is working again.