ReportAgent restart due to memory threshold breach 300MB

Products

CA Virtual Privilege Manager CA Privileged Identity Management Endpoint (PIM)

Issue/Introduction

PIMv14 agents running on RHEL6.10 and RHEL 7.5 are showing event messages on our endpoints.

This can be seen when you query for a deployed policy on the endpoint using the selang command:-

sr hnode __local__

The observed message says:-

"The product has restarted ReportAgent due to memory threshold breach 300MB"

What does this message mean and if this is an issue what is the resolution.?

Finding from the seosdb:

====================

editres HNODE ('__local__') attributes+(REGISTERED_NAME=server.example.com MAC_ADDRESS=##-##-##-##) events+('Type=Restart;Time=1570939786;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1571025726;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1571198506;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1571112571;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1570852946;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1570767010;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1570680165;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1570594230;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1570507390;Module=ReportAgent;Reason=Memory;Count=300' 'Type=Restart;Time=1570421450;Module=ReportAgent;Reason=Memory;Count=300')

To avoid warning messages I recommend that you check how much memory is consumed by AgentManager after it initializes itself (say, 30 seconds after you see its start message) and adjust the ProcVSizeHigh token in seos.ini if its value is too close to the AgentManager virtual memory size. I use the following to check that process memory:

/tmp/CA> ps -e -o pid,comm,%cpu,vsz,rss | egrep 'AgentManager|ReportAgent'
128605 ReportAgent 0.0 162520 3804
129017 AgentManager 0.0 257756 15768 <--- Here it is 257MB (without plugins).and it should be OK for the limit of 300

With PUPMAgent configured in, it becomes

/tmp/CA> ps -e -o pid,comm,%cpu,vsz,rss | egrep 'AgentManager|ReportAgent'
129771 ReportAgent 0.0 162520 3800
130182 AgentManager 0.0 389504 14584
130200 AgentManager 0.1 536132 20024 <--- the default of 300 MB is inadequate for this configuration and needs to be increased to avoid seoswd repeatedly logging message about a process that went over the memory limit

Environment

Release : 14.0

Component : CA ControlMinder

Cause

Because of the large installation and handling of the data, Reportagent is crossing the memory threshold and causing it to restart.

Resolution

Increase seos.ini token ProcVSizeHigh to 600.

Additional Information

The threshold of 300 MB was chosen fairly arbitrarily because

It was based on observations in a small-scale setup;
It was initially the same across all supported platforms. The idea behind that token (and it was a standard approach in eTrust division, e.g., uxauthd in UNAB has it even though it does not need that now (after memory leaks were eliminated)) is to restart a long-running AC process if it seems to leak memory as reflected by its reaching a certain size in memory that is deemed excessive. Thus, the 300 MB limit may not be appropriate for a large installation and needs to be raised in that case. One can increase that limit to say 600 or even 800 (MB) without any impact on performance and if the new value is still lower than ReportAgent actually needs for its normal operation, it should be adjusted again.