'Memory cgroup out of memory: Killed process <pid> (python3.10)’ message appeared on the console of a specific platform node.
search cancel

'Memory cgroup out of memory: Killed process <pid> (python3.10)’ message appeared on the console of a specific platform node.

book

Article ID: 415983

calendar_today

Updated On:

Products

VCF Operations for Networks

Issue/Introduction

It is observed in /var/log/arkin/upstart/policy-manager.log that OOM happens repeatedly.

<DATE_TIME> platform6 systemd[1]: policy-manager.service: A process of this unit has been killed by the OOM killer.
<DATE_TIME> platform6 systemd[1]: policy-manager.service: A process of this unit has been killed by the OOM killer.
<DATE_TIME> platform6 systemd[1]: policy-manager.service: A process of this unit has been killed by the OOM killer.
<DATE_TIME> platform6 systemd[1]: policy-manager.service: A process of this unit has been killed by the OOM killer.
<DATE_TIME> platform6 systemd[1]: policy-manager.service: A process of this unit has been killed by the OOM killer.
<DATE_TIME> platform6 systemd[1]: policy-manager.service: A process of this unit has been killed by the OOM killer.

Environment

VCF Operations for Networks

Cause

This issue is caused by memory pressure as there are lots of virtual devices and interfaces.

Resolution

As a workaround, you could choose one of the following two options.

1. Disabling NAV Intents as shown below could be considered if physical devices are not used.

HSRP/VRRP Configuration Error

Duplex Mismatch

Native VLAN Tagging Mismatch

Native VLAN Mismatch

Port Channel Member Mismatch

HSRP/VRRP Active STP Root Colocation

STP Path Cost Method Inconsistency

MTU mismatch

Trunk VLAN mismatch

Port mode mismatch

Duplicate MAC Address

Duplicate IP Address

Loop Detection

 

2. Increasing the memory limit of policy-manager service could be considered if physical devices are used.

# ub

// Change MemoryLimit value from 1G to 8G in the following file
# vi /home/ubuntu/build-target/policy-manager/policy-manager.service

// Restart policy-manager service
# sudo service policy-manager stop
# sudo service policy-manager start