Unable To Add Secondary Node To Primary Node, Getting "Authentication Failed" Error

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

Get authentication failure when attempting to add a secondary node to primary node: 

paramiko.ssh_exception.AuthenticationException: Authentication failed

Environment

• EDR Server: 7.4.1 and Higher

Cause

• This error means that the primary server cannot connect using SSH keys to the newly added secondary node.
• This error will occur if the primary sever fails to write an SSH key to the secondary server when the "/usr/share/cb/cbcluster add-node" command is run,

Resolution

1. Log on to the CLI of the primary server as root. 
2. Get the SSH key from the primary server. Run:

   cat /etc/cb/cb_ssh.pub

3. Look for the key entry line that begins with ssh-rsa and ends with CarbonBlack.
4. Open a second SSH terminal window to the new secondary server that you are trying to add.
5. On the secondary server, edit: /root/.ssh/authorized_keys
6. Copy and paste the SSH key line from step 3 onto the bottom of the secondary server's authorized_keys file
7. Save the file
8. Ensure that the cluster is fully stopped
9. Try re-adding the new node with the "add node" command

Additional Information

In the unlikely event that the /etc/cb/cb_ssh.pub file does not exist on the primary server, follow these steps:

1. Ensure you are logged in to the primary server as root
2. Edit: /etc/cb/cb.conf
3. Look for this line and add it to the file if it does not exist:

sshPrivateKey=/etc/cb/cb_ssh if it doesn't exist

4. Save the file.
5. Run:

ssh-keygen -t rsa

6. A prompt will appear: "Enter file in which to save the key". Enter:

/etc/cb/cb_ssh

(Note: do not add .pub, it will create a public and private key itself)

7. Press enter twice, do not put a password in
8. Run: 

/usr/bin/ssh-copy-id -i /etc/cb/cb_ssh.pub root@<ip of minion> -p 22

This last step will copy the key over. If it doesn't then manually copy it manually into /root/.ssh/authorized_keys on the secondary server by following the Resolution steps above.