UIM java probes - guidance on configuring java heap min max memory settings
search cancel

UIM java probes - guidance on configuring java heap min max memory settings

book

Article ID: 395464

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM) CA Unified Infrastructure Management On-Premise (Nimsoft / UIM) CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

  • DX UIM (Infrastructure Observability) java probes contain default configuration values for Java min/max heap memory
  • The OOTB, default settings are based on a medium-scale environment
  • This article discusses how to assess the need for increasing the min/max values when it is not obvious from the log entries, errors generated, or poor probe or webapp performance

Environment

  • DX UIM 23.4 CU3 or higher
  • Any java probe, e.g., snmpcollector, wasp, oi_connector, apm_bridge, vmware, etc.

Cause

  • Guidance

Resolution

UIM Java probes memory settings

  • The java probe min/max values are always based on the UIM environment needs.

  • Broadcom engineering provides certain defaults based on medium-scale usage.

  • The Java settings are quickly accessible via Raw configure for any probe under the startup->opt section but in some cases where that location doesn't exist, they would need to be adjusted in the probe java arguments.

  • Please note that you must check to see if you have enough available physical memory before increasing the max value


Large Environments

  • For high-scale usage in large environments, adjustments may need to be made to the min/max values as the probe workload increases due to increases in the number of monitored systems, devices, and/or targets.
  • Note that after making any min/max memory changes, the probe should be cold started (Deactivate-Activate).

    • Deactivate the probe
    • Wait until the port and PID disappear from the IM/AC console
    • Then Activate
    • Check the memory via Task Manager or top command for before/after comparison


Performance and Stability

  • Performance and stability depend on sufficient memory and resource allocation. With snmpcollector, when the number of devices/components exceeds 500,000 (which can happen with a large number of devices and metrics), a customer might experience issues like the probe turning red, inability to open the probe configuration, performance degradation, and perhaps even PPM-### failures or errors.

  • To address this, we can increase the probe's minimum and maximum memory settings (e.g., 10 and 12 GB) or deploy additional snmpcollector instances on separate hubs and 'split up' / distribute the load. But aside from that, the most obvious indication is when a java probe log throws an OutOfMemory error. Shown below is an example java error:
java.lang.OutOfMemoryError: GC overhead limit exceeded
  • This may happen with the wasp probe at times, and you can see the 'memory used' versus 'memory available' in the wasp.log at loglevel 5. In that case, the log is helpful info for setting the min-max plus we recommend adding some 'breathing room' for memory to the max value.

  • Note also that the difference between the min and the max values should never be more than 2 GB. If the min and max values are far apart, GC performance can suffer.

  • One easy way of tracking java memory usage in UIM is to add "process" monitoring using the processes probe to monitor java memory usage over time to determine if the max settings are sufficient or not.  Once the java probe process monitoring is implemented, after a few hour/days you can begin to examine the trend, min, max, std dev., etc.

  • Then use the Operator Console (OC) Performance Reports Designer (PRD) to create a report.

  • If the heap is rising but seems unreasonably high, it may need a memory 'profiling' and recheck from our Development team so please make support aware if that is the case.

  • If you see a java probe using increasingly more memory, there could be a slow memory leak. Monitoring the probe process will reveal if that is the case. Memory leaks may also adversely impact CPU usage.

GC Improvement

These arguments can be added to the start or end of any java probe startup options.

   -XX:+UseG1GC -XX:MaxGCPauseMillis=10

In Java 9 or higher, there is no need to add these arguments as Java 9 uses G1GC.

Advantages of using G1 GC

  • Improved Pause Times

    G1 GC aims to provide more predictable and shorter pause times, especially for applications with large heaps, by minimizing the duration of stop-the-world garbage collection pauses.

  • Adaptive Behavior

    G1 GC adjusts its heap partitioning and collection strategies based on the application’s workload, allowing it to dynamically adapt to changing memory usage patterns.

  • Better Utilization of Hardware Resources

    G1 GC utilizes multiple threads for parallel garbage collection, which can lead to better utilization of available CPU resources, resulting in improved application throughput.

  • Reduced Fragmentation

    G1 GC’s compacting algorithm reduces memory fragmentation by reclaiming memory from regions with a high garbage-to-live object ratio.

Powered by Wolken Software