Performance Management Data Repository permissions denied errors

Products

• CA Infrastructure Management
• CA Performance Management
• Network Observability

Issue/Introduction

When running a vbr copycluster task on the Performance Management Data Repository Vertica database it fails with an error.

The error in the vbr log files will state:

• Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

The error may also be seen when trying to run dr_validate. The cause and solution is the same as that documented below.

Environment

All supported Performance Management releases

Cause

Use of copy and replace commands which replace what may already exist, instead of concatenate commands to append what may already exist.

When the source cluster is already set up with SSH keys, following the documentation could lead to an overwrite of existing SSH keys files on the source.

As a result of the overwrite it breaks the SSH keys required for successful passwordless SSH functionality.

At this point when the SSH function used by the copycluster functionality runs it throws the error noted.

This is an error that would not have been seen through simple ssh command tests between cluster hosts.

Resolution

The documentation for SSH configuration and set up will be changed to show the use of the following solution.

The change here is using the cat command instead of the cp command previously included in the documentation.

This allows the change to be appended to existing files, or new files written if not present and avoids file overwrite problems.

We need to first check the home directory for the root, sudo root or dradmin OS user involved. We're seeking whether or not these two files exist in the users ~/.ssh directory.

• id_rsa
• id_rsa.pub

The simple method to check, without having to know the users home directory, is the following.

1. Log in to the host in a terminal window as the target user.
2. Run the command which will list (ls) all files with ownership and permissions (-al) in the users home directory (~) in the hidden .ssh directory. 
   • ls -al ~/.ssh
3. Sample output for a root user, and for a default example dradmin user.

[root@node0001~]# ls -al ~/.ssh

total 24

drwx------. 2 root root  132 Sep 19  2019 .

dr-xr-x---. 7 root root 4096 Sep 20  2019 ..

-rw-------. 1 root root 1251 Sep 19  2019 authorized_keys

-rw-r--r--  1 root root  417 Sep 19  2019 authorized_keys2

-rw-------. 1 root root    0 Sep 17  2019 authorized_keys.orig

-rw-------  1 root root 1675 Sep 19  2019 id_rsa

-rw-r--r--  1 root root  417 Sep 19  2019 id_rsa.pub

-rw-r--r--  1 root root  753 Sep 19  2019 known_hosts

[root@node0001~]# su dradmin

[dradmin@node0001root]$ ls -al ~/.ssh

total 20

drwx------ 2 dradmin verticadba  104 Sep 20  2019 .

drwx------ 6 dradmin verticadba  177 Sep 30  2019 ..

-rw-r--r-- 1 dradmin verticadba  381 Sep 20  2019 authorized_keys

-rw-r--r-- 1 dradmin verticadba  381 Sep 20  2019 authorized_keys2

-rw------- 1 dradmin verticadba 1679 Sep 20  2019 id_rsa

-rw------- 1 dradmin verticadba  381 Sep 20  2019 id_rsa.pub

-rw-r--r-- 1 dradmin verticadba  346 Sep 20  2019 known_hosts

Scenario 1: The id_rsa and id_rsa.pub files are not present in the users ~/.ssh directory.

If the id_rsa and id_rsa.pub files are not present in the users ~/.ssh/ directory perform the following steps.

1. Run these three commands:
   • ssh-keygen -N "" -t rsa -f ~/.ssh/id_rsa
   • cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys2
   • chmod 644 ~/.ssh/authorized_keys2
   • cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
   • chmod 644 ~/.ssh/authorized_keys
2. Run the following, once each for all hosts you want to log into.
   • ssh-copy-id -i user@remotehost

Scenario 2: The id_rsa and id_rsa.pub files are already present in the users ~/.ssh directory.

If the id_rsa and id_rsa.pub files are present in the users ~/.ssh/ directory perform the following steps.

1. Run these commands:
   • cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
   • cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys2
2. Run the following to all hosts you want to log into.
   • ssh-copy-id -i user@remotehost