CPU 100% usage causing OOM errors for Capacity (CPA) pods.

Products

• DX Operational Observability

Issue/Introduction

Cpa-projection pod has been found with 100% CPU usage for extended period leading to the pod crashing with OOM error and restarting when over 700 services have been assigned to default capacity (CPA) group.

 

Environment

DXO2 on-prem.

Resolution

Scale up cpa-projection deployment to few instances, from 2 to 4 instances so the overhead will be dispatched by the command shown below.

a) For Kubernetes: kubectl scale deployment cpa-projection-001 --namespace [namespace where deployment resides] --replicas 2

b) For Openshift: oc scale deployment cpa-projection-001 --namespace [namespace where deployment resides] --replicas 2

 

In case scalling up to 4 instances is not helping Contact Broadcom Support.