The Redis Service crashing due to almost 100% memory utilization

Products

• VMware Aria Suite

Issue/Introduction

Redis Service crashing due to nearly 100% Memory Utilization.

RaaS server is logging messages related to command compilation.

2024-08-23 14:22:32,311 [var.lib.raas.unpack._MEIWmzae7.raas.mods.db.schedule              ][WARNING :216 ][ForkPoolWorker-1:4385] Compile commands warn threshold (1000) exceeded. Stats = {'cmd_compile': 823387}
2024-08-23 14:22:32,311 [var.lib.raas.unpack._MEIWmzae7.raas.mods.db.schedule              ][WARNING :216 ][ForkPoolWorker-7:4421] Compile commands warn threshold (1000) exceeded. Stats = {'cmd_compile': 1083835}
2024-08-23 14:22:32,311 [var.lib.raas.unpack._MEIWmzae7.raas.mods.db.schedule              ][WARNING :216 ][ForkPoolWorker-3:4397] Compile commands warn threshold (1000) exceeded. Stats = {'cmd_compile': 807757}
2024-08-23 14:22:32,312 [var.lib.raas.unpack._MEIWmzae7.raas.mods.db.schedule              ][WARNING :216 ][ForkPoolWorker-6:4415] Compile commands warn threshold (1000) exceeded. Stats = {'cmd_compile': 444782}

Environment

Aria Automation Config 8.x

Cause

The Redis server is using nearly 100% of its memory.

The Redis database has grown to 17 GB, and the Celery queue depth is increasing exponentially.

Resolution

This needs to be addressed at the database level. Please submit a support case by following the instructions in the KB: https://knowledge.broadcom.com/external/article/142884/creating-and-managing-broadcom-support-c.html.