Block federation publishers if there're RabbitMQ memory/disk alarms
Article ID: 416404
Updated On:
Products
Issue/Introduction
In RabbitMQ, publishers are normally blocked when memory or disk alarms are triggered to prevent system overload. However, users observed that federation publishers were not being blocked under such conditions. This raised concerns about inconsistent alarm handling between federation links and other publishing mechanisms such as shovels.
Environment
RabbitMQ 4.2.X
Cause
Initially, RabbitMQ did not enforce alarm-based publishing blocks for all components.
The earlier implementation only affected certain clients, meaning federation links could continue publishing messages even when the broker was under memory or disk pressure.
RabbitMQ issue #14657 (“Block direct shovel publishes during alarms”) introduced a change to honour alarms for direct client connections.
Although the title of the issue mentions shovels, the fix applies more broadly.
“#14657 applies to everything that uses the direct Erlang client: shovel, federation, logging to amq.rabbitmq.log… Shovel and federation do not always use the direct client. For example, for connections to remote clusters that’s not the case because the direct client usually wouldn't be able to authenticate with them.”
Therefore, federation links that use direct connections (typically within the same cluster via amqp(s):// URIs) now respect memory/disk alarms, while remote federation links using normal network connections may not.
Resolution
This behaviour has been addressed in RabbitMQ 4.2.1 and later, as part of the fix for issue #14657.
-
Shovels and federation links that use the direct Erlang client will now honour memory and disk alarms.
-
Remote federation links (to external clusters) may still not be blocked, as they typically use network-based AMQP connections rather than the direct client.
Additional Information
Feedback
