CAPC does not connect to DA after being upgraded from release 3.6.x/3.7.x to 21.2.1 or later

Products

DX NetOps CA Infrastructure Performance CA Performance Management - Usage and Administration

Issue/Introduction

The PC (Performance Center) is unable to contact the DA (Data Aggregator) after upgrading from 3.6.x or 3.7.x to 21.2.1 or later release.

We are following the steps outlined in the following KB article:

Data Aggregator will not sync after Netops Portal upgrade to version 21.2.2 or later
https://knowledge.broadcom.com/external/article?articleId=223381

But unable the get the RESTful output via curl on the DA machine (you can replace the admin account with another one with the Administrator role).

HTTP/1.1 403 Forbidden

$ curl -u admin -v -s -k -X GET http://localhost:8581/rest/dataaggregator

$ curl -u admin -v -s -k -X GET http://localhost:8581/rest/

Environment

DX NetOps CAPM Release : 21.2

Component : IM Data Aggregator

Cause

There were two problems:

SsoEncryptionDecryptionsKey mismatch between PC (MySQL) and DR (Vertica)
NpcHostName has an invalid hostname in DR (Vertica)

If the NpcHostName in DA for 3.6 or 3.7 is not resolvable or reachable, we can't validate the REST credentials and allow the user to update via REST.

We have no way to update DA except via REST. But that assumes DA can reach PC correctly.

On the DR - Vertica machine:

Gather the SsoEncryptionDecryptionsKey and NpcHostName from the Vertica database to cross check with PC.

$ cd /opt/vertica/bin

Login as dauser NOT dradmin

$ ./vsql -U dauser -W (prompt for password)

 select item_id,attr_qname,string_value from dauser.v_attribute_instance where attr_qname like '%SsoEncryptionDecryptionKey';

item_id | attr_qname | string_value
---------+--------------------------------------------------------------------+--------------
8 | {http://im.ca.com/da}DataAggregatorInfo.SsoEncryptionDecryptionKey | #$utP9%z
(1 row)

 select item_id,attr_qname,string_value from dauser.v_attribute_instance where attr_qname like '%NpcHostName';

item_id | attr_qname | string_value
---------+-----------------------------------------------------+---------------
8 | {http://im.ca.com/da}DataAggregatorInfo.NpcHostName | https
(1 row)

Note down the item_id, in this case, 8. It will be used later to identify the specific DA linked attribute to be updated.

On the PC machine:

Gather the SsoEncryptionDecryptionsKey from the MySQL database.

$ mysql -uroot -p (prompt for password)

You may have different errors in the log files:

========================================================

In the /opt/IMDataAggregator/apache-karaf-4.2.6/data/log/karaf.log file (included in the DA remote engineer output)

In the /opt/IMDataAggregator/apache-karaf-4.2.6/data/log/AuthenticationLog.log file (included in the DA remote engineer output)

The Web Service Host value is https which is not a valid hostname for the PC machine.

Resolution

We will update the Vertica database with the SsoEncryptionDecryptionsKey value from the MySQL database and the PC hostname (manually).

1. First, stop the DA service on the DA machine.

$ service dadaemon stop

2. Go to the DR machine and update the following attributes in Vertica. Note, use the item_id as found above in the previous select statement on the DR, in this example, 8.

$ cd /opt/vertica/bin

$ ./vsql -U dauser -W (prompt for password)

This query will update the PC hostname and the SsoEncryptionDecryptionKey at the same time:

Please replace ‘correctPChostname.whatever.org’, ‘a1P4UMWW’, and 8 with the correct values for your environment.

update attribute_instance set string_value='correctPChostname.whatever.org' where item_id=8 and attribute_id=(select attribute_id from v_attribute where attr_qname like '%}DataAggregatorInfo.NpcHostName' limit 1);update attribute_instance set string_value='a1P4UMWW' where item_id=8 and attribute_id=(select attribute_id from v_attribute where attr_qname like '%}DataAggregatorInfo.SsoEncryptionDecryptionKey' limit 1);commit;

If only one of these values needs to be changed:

This query only updates the PC hostname:

dauser=> update attribute_instance set string_value='PC_hostname' where item_id=8 and attribute_id=(select attribute_id from v_attribute where attr_qname like '%}DataAggregatorInfo.NpcHostName' limit 1);

dauser=> commit;

This query only updates the SsoEncryptionDecryptionKey:

dauser=> update attribute_instance set string_value='a1P4UMWW' where item_id=8 and attribute_id=(select attribute_id from v_attribute where attr_qname like '%}DataAggregatorInfo.SsoEncryptionDecryptionKey' limit 1);

dauser=> commit;