How does CDM probe calculate memory usage on ALL OS systems

Products

DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

What formula does the cdm probe use to calculate memory usage?

How does CDM calculate memory usage on the OS systems?

Environment

DX UIM 20.4.x / 23.4.x

CDM probe 7.x or later

Resolution

Linux:

=========

Cdm reads the proc/meminfo file to obtain the memory usage qos data. For example:

# cat /proc/meminfo
MemTotal:        8175444 kB
MemFree:          154404 kB
Buffers:               4 kB
Cached:          1978176 kB
SwapTotal:       8257532 kB
SwapFree:        2962720 kB

The probe uses the following formulas for the different qos metrics:

QOS_MEMORY_TOTAL (Total Memory MB) = (MemTotal + SwapTotal) / 1024 QOS_PHYSICAL_MEMORY_TOTAL (Physical Memory MB) = MemTotal / 1024

QOS_MEMORY_SWAP (SwapUsed MB) = (SwapTotal - SwapFree) / 1024
QOS_MEMORY_USAGE (Total Memory Used MB) = (MemTotal - MemFree - Buffers - Cached + SwapUsed) /1024
QOS_MEMORY_PHYSICAL (Physical Memory Used MB) = (MemTotal - MemFree - Buffers - Cached) /1024

QOS_MEMORY_PERC_USAGE (Total Memory Used %) = (Total Memory Used/Total Memory) * 100
QOS_MEMORY_PHYSICAL_PERC (Physical Memory Used %) = (Physical Memory Used/Physical Memory) * 100

Since cdm v5.40, the QOS_MEMORY_PHYSICAL metric, by default, excludes the buffer and cache.

If you follow the resolution in https://knowledge.broadcom.com/external/article?articleId=47141 to include the buffer and cache, the calculation for the metric is

QOS_MEMORY_PHYSICAL (Physical Memory Used MB) = (MemTotal - MemFree) /1024

Solaris:

============

To calculate Physical Memory Usage (“QOS_PHYSICAL_MEMORY_PERC”) on Solaris the probe does not use any tool or utility ( "prstat -Z") but it uses APIs to get the physical memory information.

Get the total physical memory:

Phys Mem Pages = sysconf(_SC_PHYS_PAGES)

Total Physical Memory = multiplied by the Page Size * number of Mem Pages

Get physical available memory information:

Phys Avail Mem Pages = sysconf(_SC_AVPHYS_PAGES);

Availabe Physical Memory = multiplied by the Page Size * number of Phys Avail Mem Pages

Get physical memory usage information:

Physical Memory Used = Total Physical Memory - Availabe Physical Memory

QOS_MEMORY_PHYSICAL (Physical Memory Used) = Physical Memory Used – Data Buffer and Cache

Get physical memory usage percentage information:

QOS_MEMORY_PHYSICAL_PERC (Physical Memory Used %) = (Physical Memory Used/Total Physical Memory) * 100

Note:

Buffers and Cached memory (zfs for global and local zone) are counted as free unless the customer sets mem_buffer_used=yes

If set for mem_buffer_used=no
Formula: Used Physical - ( ZFS File Data + Exec and libs + Page cache)

Free (cachelist) is not included in formula due fixed issue CDM probe (on Solaris) reporting negative memory values))

So the QOS_MEMORY_PHYSICAL metric, by default, excludes the buffer and cache.

Additional Information

The meminfo file reports the memory in KB and the probes reports the QOS Metric in MB hence we divide by 1024 in the above formulas.
You will very likely see a small difference in the values in meminfo and the QOS metric. Memory usage is a constantly shifting value and the probe will have collected at a slightly different time to when you viewed the meminfo file.
Swap usage is found using the system call swapctl().
Paging data is found by running either “/usr/bin/vmstat –s” or “/usr/5bin/vmstat –s” (depending on which is installed on the system) and parsing the data it provides.

Article title: How does CDM calculate Memory Usage in Solaris?

Article title: How is Memory information calculated by cdm probe for HP-UX based systems

Article title: How Is "Swap Usage" activity calculated by the cdm probe in Solaris

Article title: How Is "memory paging" activity calculated by the cdm probe?

Article Title: Monitoring Total Processor / CPU utilization on Windows with UIM