EDR: Error: RabbitMQ Crash {{nocatch,{bad_argument,type}}

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• /var/log/cb/rabbitmq/log/crash.log:

2019-11-01 04:09:25.405 [error] emulator Error in process <0.31240.2> on node 'rabbit@CB-SERVER-CLUSTER-HEAD-NODE' with exit value:
{{nocatch,{bad_argument,type}},[{rabbit_amqqueue,info,2,[{file,"src/rabbit_amqqueue.erl"},{line,751}]},{rabbit_misc,with_exit_handler,2,[{file,"src/rabbit_misc.erl"},{line,507}]},{rabbit_control_misc,step_with_exit_handler,4,[{file,"src/rabbit_control_misc.erl"},{line,67}]},{rabbit_control_misc,'-emitting_map0/5-lc$^0/1-0-',5,[{file,"src/rabbit_control_misc.erl"},{line,59}]},{rabbit_control_misc,emitting_map_with_exit_handler,5,[{file,"src/rabbit_control_misc.erl"},{line,55}]},{rabbit_control_misc,emitting_map_with_exit_handler,4,[{file,"src/rabbit_control_misc.erl"},{line,50}]}]}

2019-10-31 00:12:47.640 [info] <0.23767.2> connection <0.23767.2> (127.0.0.1:54678 -> 127.0.0.1:5004): user 'cb' authenticated and granted access to vhost '/'

2019-10-31 00:12:47.893 [warning] <0.23767.2> closing AMQP connection <0.23767.2> (127.0.0.1:54678 -> 127.0.0.1:5004, vhost: '/', user: 'cb'):
client unexpectedly closed TCP connection

 

• /var/log/cb/rabbitmq\[email protected]:

2019-11-01 04:09:25.405 [error] emulator Error in process <0.31240.2> on node 'rabbit@CB-SERVER-CLUSTER-HEAD-NODE' with exit value:
nocatch,{bad_argument,type,[{rabbit_amqqueue,info,2,[

{file,"src/rabbit_amqqueue.erl"},{line,751}]},{rabbit_misc,with_exit_handler,2,[{file,"src/rabbit_misc.erl"},{line,507}]},{rabbit_control_misc,step_with_exit_handler,4,[{file,"src/rabbit_control_misc.erl"},{line,67}]},{rabbit_control_misc,'emitting_map0/5-lc$^0/1-0',5,[{file,"src/rabbit_control_misc.erl"},{line,59}]},{rabbit_control_misc,emitting_map_with_exit_handler,5,[{file,"src/rabbit_control_misc.erl"},{line,55}]},{rabbit_control_misc,emitting_map_with_exit_handler,4,[{file,"src/rabbit_control_misc.erl"},{line,50}]}]}
/rabbitmq log
2019-11-01 04:39:53.494 [error] emulator Error in process <0.31423.2> on node 'rabbit@CB-SERVER-CLUSTER-HEAD-NODE' with exit value:
{{nocatch,{bad_argument,type}},[{rabbit_amqqueue,info,2,[{file,"src/rabbit_amqqueue.erl"},{line,751}]},{rabbit_misc,with_exit_handler,2,[{file,"src/rabbit_misc.erl"},{line,507}]},{rabbit_control_misc,step_with_exit_handler,4,[{file,"src/rabbit_control_misc.erl"},{line,67}]},{rabbit_control_misc,'-emitting_map0/5-lc$^0/1-0-',5,[{file,"src/rabbit_control_misc.erl"},{line,59}]},{rabbit_control_misc,emitting_map_with_exit_handler,5,[{file,"src/rabbit_control_misc.erl"},{line,55}]},{rabbit_control_misc,emitting_map_with_exit_handler,4,[{file,"src/rabbit_control_misc.erl"},{line,50}]}]}
2019-11-01 04:42:43.572 [info] <0.31441.2> accepting AMQP connection <0.31441.2> (127.0.0.1:34582 -> 127.0.0.1:5004)
2019-11-01 04:42:43.575 [info] <0.31441.2> connection <0.31441.2> (127.0.0.1:34582 -> 127.0.0.1:5004): user 'cb' authenticated and granted access to vhost '/'
2019-11-01 04:42:43.828 [warning] <0.31441.2> closing AMQP connection <0.31441.2> (127.0.0.1:34582 -> 127.0.0.1:5004, vhost: '/', user: 'cb'):

Environment

• EDR Server: 6.4.x - 6.5.2

Cause

• Known Issue in the RabbitMQ 3.7.14 Build
  • This known issue is fixed in the RabbitMQ 3.7.16 Build. 
  • RabbitMQ will be updated to 3.7.19 or later in the EDR Server 7.0.0 Release.
• Text from RabbitMQ Bug Report:
  • Type is a new argument for 3.8.x. Listing it here causes rabbitmqctl report to
    crash with {nocatch,{bad_argument,type}} when it tries to list queues with all
    arguments.
• This crash report should not affect the normal functionality of the RabbitMQ bus and event flow should not be affected.

Resolution

• Workaround: 
  • When using /usr/share/cb/cbrabbitmqctl use the commands below instead of the report argument:

status
environment
list_connections
list_channels
list_queues
list_bindings
list_ciphers
list_consumers
list_exchanges