Error: RabbitMQ container crashes with Exit Code 137 in API Developer Portal 5.4.x

Products

• CA API Developer Portal

Issue/Introduction

In the API Developer Portal v5.4.x, activation and reactivation emails are not being sent to users. This occurs because the portal_rabbitmq container crashes intermittently, preventing the portal_dispatcher service from processing the notify.queue

Symptoms

• Activation or password reset emails are not received.
• rabbitmqctl list_queues shows messages accumulating in notify.queue with 0 consumers.
• RabbitMQ container logs show Exit Code 137.
• Running docker inspect #### --format "{{.State.OOMKilled}}" returns true.

Environment

• API Developer Portal 5.4.1 and higher
• Docker Swarm Deployment

Cause

Following the removal of the PSSG service, the portal_dispatcher handles notification consumption. The default memory limits for the RabbitMQ (message-broker) container (560 MiB) are insufficient for production loads, leading to Out of Memory (OOM) termination by the Docker daemon.

Resolution

Increase the memory allocation for the RabbitMQ service in the Docker deployment configuration.

1. Open the docker-compose.yml file used for the Portal deployment.
2. Locate the rabbitmq service definition.
3. Update the resources section under deploy to increase memory limits:
   yaml

   rabbitmq:  
       deploy:
         replicas: 1
         placement:
           constraints:
             - node.labels.portal == true
         update_config:
           parallelism: 1
           delay: 10s
           failure_action: pause
         resources:
           limits:
             memory: 1024M
           reservations:
             memory:768M
         restart_policy:
           condition: any

4. Redeploy the stack to apply changes: docker stack deploy -c docker-compose.yml portal
5. Verify the portal_rabbitmq service is stable and the notify.queue has an active consumer: docker exec #### rabbitmqctl list_queues name messages consumers