Cpa-projection Pod CrashLoopBackOff: OOMKilled with Exit Code 137 due to large capacity group.

Products

• DX Operational Observability

Issue/Introduction

The cpa-projection pod is failing intermittently or entering a CrashLoopBackOff state. It has been found that default capacity group has over 600 services assigned. When inspecting the pod's status, the container reports being terminated by the system due to memory exhaustion.

Horizontal scaling (increasing the replica count) of the deployment does not resolve the issue.

Running kubectl describe pod <cpa-projection-pod-name> reveals the following termination state:

Last State:     Terminated                                                                                                          
  Reason:        OOMKilled                                                                                                            
  Exit Code:     137      

Resolution

 

1. Edit the cpa-projection deployment:

   kubectl edit deployment cpa-projection
   

2. Locate the resources section under the container spec and update the limits and requests to match the configuration below:

   resources:
     limits:
       cpu: "4"
       memory: 8Gi
     requests:
       cpu: 100m
       memory: 512Mi
   

3. Save and exit the editor. Kubernetes will trigger a rolling restart of the pod with the newly allocated resources.

4. Verify that the new pod stabilizes and no longer enters an OOMKilled state:

   kubectl get pods -w | grep cpa-projection