ABX action are getting failed interminttent

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

ABX actions may intermittently fail to send confirmation back to the running deployment even though the action itself completes successfully.

For example, during VM creation, an ABX action is triggered to assign a custom name based on the selected options. The action successfully generates and applies the name, but the confirmation is not always received by the deployment, causing it to remain in progress or fail.

Environment

Aria Automation 8.18.x

Cause

The issue occurs when quorum queues in RabbitMQ do not have an assigned leader. RabbitMQ is used by the Event Broker Service as the underlying message broker.

The following entries may be observed in the RabbitMQ logs:

2025-09-05 14:02:04.550726+00:00 [info] <0.10142.1> queue 'com.vmware.automation.catalog-idem-events-subscription-idem-resources-provider-q###33c3-433#######fd7291' in vhost '/': leader call - leader not known. Command will be forwarded once leader is known.
2025-09-05 14:02:04.553645+00:00 [info] <0.11007.1> queue 'com.vmware.automation.relocation-service-deployment-resource-action-post-qq-d4c6####-186e-4bc#########d2b9' in vhost '/': leader call - leader not known. Command will be forwarded once leader is known.
2025-09-05 14:02:05.328961+00:00 [info] <0.24779.1> queue 'com.vmware.automation.broker.broadcast.command-qq-55ce9567-#######57541da' in vhost '/': leader call - leader not known. Command will be forwarded once leader is known.
2025-09-05 14:02:13.069557+00:00 [info] <0.14993.2> queue 'com.vmware.automation.broker.broadcast.command-qq-88bd########916ce800' in vhost '/': leader call - leader not known. Command will be forwarded once leader is known.

These messages indicate that the queues are in follower state without a leader, resulting in communication delays or failures.

Resolution

To resolve the issue, identify the queues missing a leader and manually trigger a leader election for them using the following command:

kubectl exec rabbitmq-ha-0 -n prelude -- rabbitmqctl eval 'V = <<"/">>, QName = <<"test-qq">>, {ok, QRec} = rabbit_amqqueue:lookup(rabbit_misc:r(V, queue, QName)), RaID = amqqueue:get_pid(QRec), ra:trigger_election(RaID).'

Note: Replace test-qq with the actual queue name from the RabbitMQ logs.

If you are unsure which queues are missing leaders or need assistance analyzing the RabbitMQ logs, raise a support request with Broadcom Support to have the logs reviewed.