RabbitMQ appears to not exchange the notifications from the SAM to MnR (post rabbitmq repository cleanup) with an error like:
ERROR 'ALERTS-IN' EventConsumerService alerts-ingestion-main AMQP connection failure ERROR 'ALERTS-IN' EventConsumerService alerts-ingestion-main AMQP connection failure ERROR 'ALERTS-IN' EventConsumerService alerts-ingestion-main start java.lang.RuntimeException: No available ingestion Queue
VMware Smart Assurance Service Assurance Manager - 10.1.X
Watch4net / M&R 7.x
1). The repository cleanup of the rabbitmq means that there is no existing configuration, and hence the sm_rabbitmq.pl script reports a failure in merge operation as follows:
Waiting for RabbitMQ to be available: . OK Merging default configuration: Failed! 204 No Content
2). Further, we can see that the script is written to close operation on encountering a failure and hence the script closes on line: 205
Died at /apg/InCharge10/SAM/smarts/perl/lib/setup_rabbitmq.pl line 205.
3). Although, we can see that the RabbitMQ Queues and Users are created on the RabbitMQ console, a crucial piece of code remains un-executed as the script exits at line #205.
Hence, the visual queues for the same are that the Consumers appear to be un-initialized. This can be validated via:
http://<RabbitMQ_hosted_IP>:15672/api/consumers
The notification exchange is affected means that the notification flow to the M&R is affected and no data will be exchanged with this current status.
If you are to clean-up the rabbitmq repository ensure that the setup_rabbitmq.log does not report the mentioned error:
Merging default configuration: Failed! 204 No Content
If the above error is seen, we could simply restart the rabbitmmq service as during the first restart operation re-creates the repository and the second restart would be able to achieve the merge and no error is thrown for the sm_rabbitmq.pl script.
Note:
Smarts-tomcat and Presentation-SAM also need to be restarted as well, post restarting smarts-rabbitmq
Cleanup of RabbitMQ and ElasticSearch repos should be followed in the below sequence:
1) Stop smarts-rabbitmq, smarts-tomcat, smarts-elasticsearch and Presentation SAM
2) Clean-up the rmq repos by moving the contents under SAM/smarts/local/repos/rmq directory to a temporary location.
Ensure the same for Sitemod_SAM/smarts/local/repos/rmq directory
3) Clean-up elasticsearch repos by moving the contents under SAM/smarts/local/repos/elasticsearch directory to a temporary location.
Ensure the same for Sitemod_SAM/smarts/local/repos/elasticsearch
4) Start smarts-elasticsearch and ensure it's running.
5) Start smarts-rabbitmq alone and check setup_rabbitmq.log.
If it has failure messages (as described in this article) then restart the same once again.
Ensure no such failures seen in the setup_rabbitmq.log.
6) Start smarts-tomcat, wait for a while and ensure consumer is running
http://<RabbitMQ_hosted_IP>:15672/api/consumers
7) Start Presentation SAM.