Unable to add minion to cluster due to loopback misconfiguration

Products

• Carbon Black EDR

Issue/Introduction

• Unable to successfully add a minion server to the cluster. 
• Startup error seen on the affected minion:

Node[ 6] Process execution failure(75):
Node[ 6] Error: Failed to initialize erlang distribution: {{shutdown,
Node[ 6] {failed_to_start_child,
Node[ 6] net_kernel,
Node[ 6] {'EXIT',nodistribution}}},
Node[ 6] {child,undefined,
Node[ 6] net_sup_dynamic,
Node[ 6] {erl_distribution,
Node[ 6] start_link,
Node[ 6] [['rabbitmq-cli-09',
Node[ 6] shortnames],
Node[ 6] false]},
Node[ 6] permanent,1000,supervisor,
Node[ 6] [erl_distribution]}}.
Node[ 6] Traceback (most recent call last):
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-pack
Node[ 6] ages/cb/utils/exceptions.py", line 79, in decorator
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/maintenance/cbstartup/main.py", line 173, in main
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/maintenance/cbstartup/main.py", line 113, in run
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/maintenance/cbstartup/actions/init_rabbitmq.py", line 70, in execute
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/utils/rabbitmq_plugins.py", line 19, in enable
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/utils/rabbitmq_plugins.py", line 24, in _install_builtin_plugins
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/utils/rabbitmq_plugins.py", line 44, in _enable_plugin
Node[ 6] File "/usr/share/cb/virtualenv/lib/python2.7/site-packages/cb/utils/rabbitmq_plugins.py", line 61, in _run_process
Node[ 6] Exception: Process execution failure(75):
Node[ 6] Error: Failed to initialize erlang distribution: {{shutdown,
Node[ 6]
Node[ 6] {failed_to_start_child,
Node[ 6] net_kernel,
Node[ 6] {'EXIT',nodistribution}}},
Node[ 6] {child,undefined,
Node[ 6] net_sup_dynamic,
Node[ 6] {erl_distribution,
Node[ 6] start_link,
Node[ 6] [['rabbitmq-cli-09',
Node[ 6] shortnames],
Node[ 6] false]},
Node[ 6] permanent,1000,supervisor,
Node[ 6] [erl_distribution]}}.
Node[ 6]
Node[0 ] Successful cb-enterprise start on localhost

 

Environment

• Carbon Black EDR Server: Supported Versions
• Multi-node cluster

Cause

There are many potential reasons a minion cannot be added to a cluster including a misconfigured loopback address

Resolution

1. Verify that the /etc/hosts file on the minion node being added has correct syntax and address for the loopback (127.0.0.1) address.
2. Follow steps from the Installation Guide again to validate nothing was missed during cluster node add procedure.