Aria operation manager cluster shows an ad failure. Not able to access.

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Cluster is in failed state, can be seen from Aria Ops admin ui "https://ariaops.example.com/admin" 

Environment

VMware Aria Operations 8.x

Cause

In analytics logs /storage/log/vcops/log/analytics-<uuid>.log messages similar to:

ERROR [itask-thread-fsdb-data-retention-manager]  com.vmware.vcops.fsdb.FsdbDataRetentionManager.deleteResourceData - Delete resource data request is failed for resource .... :com.integrien.alive.FSDB.LowDiskSpaceException: Failed to increase file size. File /usr/lib/vmware-vcops/data/1/..../.....dat. Reason: FSDB is running low on disk space
ERROR [itask-thread-fsdb-data-retention-manager]  com.vmware.vcops.fsdb.FsdbDataRetentionManager.deleteResourceData - Delete resource data request is failed for resource .... :com.integrien.alive.FSDB.LowDiskSpaceException: Failed to increase file size. File /usr/lib/vmware-vcops/data/1/..../.....dat. Reason: FSDB is running low on disk space

Resolution

Please check the storage usage by running

df -h

If the /storage/db partition is full or higher than 90% of usage please follow kb below to add more free space 

Adding additional storage to a node in Aria Operations