Some UIM groups displayed in the Operator Console inventory are not listed in DX OI

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)
• CA Unified Infrastructure Management On-Premise (Nimsoft / UIM)
• CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

Some UIM groups displayed in the Operator Console inventory are not listed under uim.groups in DX OI under 'Manage Elements'

In DX OI->Manage Elements for Service has missing groups but the groups that are missing from the view, changes each time you view them. The first 40 out of 100, the last 40 out of 100, etc. It is not consistent. What shows/displays for the groups view is always changing.

When tested, nulls showed up and it is not known why so far in the UIM Inventory...oi_connector-side when we are filtering on an attribute of an element. The group information must be populated otherwise the group will not show up.

Environment

• oi_connector v1.52
• apm_bridge 1.09

Cause

• Group name is posting as null from apm_bridge payload to OI.
• Missing groups and topology json files in the debug logs, and this is due to huge data processing by the apm_bridge probe

Resolution

Logging

---

Please enable additional logging and share the data as described below: 

Edit the apm_bridge.cfg configuration file with a text editor, or use Raw Configure to update key/value pairs. You can update or add the following key/value pairs to enable additional logs and for troubleshooting:

<topology_service>      
save_incoming_graphs = 1
log_outgoing_request = 1
publish_config_items = 1
</topology_service> 

After restart of the apm_bridge proble, a new "debug" directory will be created that includes subfolders, each will contain json files of the inventory and topology data. 

Let it be run for approx. 2 hours and then deactivate the probe. 

Then zip and upload the entire probe directory to the support case. 

A KB Article describing the above steps with screenshots is listed below - see section on 'Enable additional logging...': 

DX OI integration with UIM - Troubleshooting

---

applied apm_bridge debug build v1.09T3

1. Fix provided to get the older CI Vertices which has more than 2 days of alive time
   
   To adjust the date we provided a new configuration parameter inventory_alive_time_days available in the setup section
   
2. Fix provided for vertex chunks payload to reduce the payload chunks based on space (1024 kb)

    Please follow the steps below to capture the desired information:

1. Deactivate the existing version of apm_bridge.

2. Go to pthe robe directory and copy the entire folder into safe location.

3. Delete the probe from IM and also delete the probe directory from file explorer.

4. Now deploy the debug build v1.09T3 to the robot. Do not activate the probe yet.

5. Go to the backup directory as mentioned in step 2 and copy the apm_bridge.cfg.

6. Update the CFG in the probes' current directory and it will overwrite the default CFG.

7. Now adjust the logsize, it should be 999999  and loglevel should be set to 5. Also verify the values shown below:

    <topology_service>
        publish_config_items = 1
        save_incoming_graphs = 1
        log_outgoing_request = 1
     </topology_service>

8. Then Activate the probe and wait for 30 mins. Verify the logs are captured for only the same duration.
   
   
9. Deactivate the probe and zip and upload entire probe directory to the support case for further review.

---

Verify the number of groups

To verify the number of groups present in the UIM, cs_ids and their associated group_ids here are the following queries. Please export the output into an Excel sheet (with headers) and upload the attachments to the case. This data will help us to validate the things on the DOI side. 

 

Query1

 

SELECT
   count(*)
FROM cm_group g

Query2

SELECT g.grp_id
,g.pgrp_id
,g.name
,g.description
,g.grp_type
,ag.account_id
,g.criteria
,g.master_element_group
,g.name_token
,g.description_token
,g.last_update
,g.active
,g.priority
,gpc.id AS configId
,gpc.version AS configVersion
,gpc.name AS configName
,gpc.description AS configDescription
,ap.auto_parent
FROM cm_group g
LEFT JOIN cm_account_group ag ON ag.grp_id = g.grp_id
LEFT JOIN cm_group_preconfigured gpc ON gpc.grp_id = g.grp_id
LEFT JOIN (
SELECT DISTINCT g.pgrp_id AS auto_parent
FROM cm_group_automatic_result gar
LEFT JOIN cm_group g ON g.grp_id = gar.grp_id
) ap ON ap.auto_parent = g.grp_id
WHERE NOT g.grp_id = 0

Query3

SELECT
    cs.cs_id
,STUFF((
        SELECT ',' + CONVERT(VARCHAR(10), gm.grp_id)
        FROM cm_group_member gm
        WHERE  gm.cs_id = cs.cs_id
        FOR XML PATH('')), 1, 1, '') AS grp_ids
,cs.ip
,cs.name
,STUFF((
        SELECT ',' + g.name
        FROM cm_group_member gm
        INNER JOIN [cm_group] g ON gm.grp_id = g.grp_id
        WHERE gm.cs_id = cs.cs_id
        FOR XML PATH('')), 1, 1, '') AS grp_names
FROM cm_group g
INNER JOIN cm_group_member gm ON g.grp_id = gm.grp_id
INNER JOIN cm_computer_system cs ON cs.cs_id = gm.cs_id
GROUP BY
    cs.cs_id, cs.ip, cs.name;

---

Troubleshooting

 

For troubleshooting, please perform the below steps:

1. Deactivate the apm_bridge

2. Set topology_service/save_incoming_graphs to 0

3. Enable topology_service/log_outgoing_request to 1

4. Activate the apm_bridge

Using the full UIMAPI response payload, we were able to successfully replicate the issue seeing null values.

The problem was due to the huge amount of group data, 3098 groups with 636 Container groups among them. 

This level of processing requires a higher recursion value to cover all the groups in the group reduction processing. The solution for this problem is to increase the recursion depth value to 1000. For this please add the below property for the apm_bridge probe under setup/inventory_service:

 grp_recursion_level  = 1000

1. Deactivate the apm_bridge probe

2. Open Raw Configure

3. Add the grp_recursion_level property with value-> 1000

4. Activate the probe. 

• IMPORTANT: If the number of groups increases more than 4000 in future, then grp_recursion_level value must also be adjusted.

• Due to the size of the payload, we have split origins across two profiles so that it would ease out the load on the processing of payloads to DX OI.

• In case of more origins getting added in the future then its better to split the profile into 3 (from 1 or 2).

• Also, the cache directory was filled up with partial chunks. To improve the performance we have deleted the unnecessary partial chunk files of 1 KB. This cleaning activity needs to be monitored from time to time so that the performance of apm_bridge is intact.

• Note: In future releases of apm_bridge there is an ongoing development feature to delete the 1KB partial chunk files automatically.

We verified the complete debug tasrequests and confimed that there are no null values observed for uim.groups parameter for each of the exported devices. 

 

Also, the page count of 17 x 100 currently seen on the DX OI environment is the accurate value.  

 

For example, when you view the tasrequest payload, for a given cs_id the groups are seen comma-separated. i,.e. a cs_id belonging to different comma-separated groups of hierarchy. The leaf-level static or dynamic independent groups are concatenated with their respective parent and grandparent groups with a pipe ( | ) delimiter as shown below

 

eg : "csId" : 421480 belongs to "uim.groups" :[ "PRODxxx|SERVABCDE|Xxlow", "PRODxxx|SERVABCDE|XXXCC" ]

        "csId" : 424967 belongs to "uim.groups" :[ "PRODxxx|SERVABCDE|Xxlow", "PRODxxx|SERVABCDE|XXXCC" ]

        "csId" :  424968 belongs to "uim.groups" :[ "PRODxxx|SERVABCDE|Xxlow", "PRODxxx|SERVABCDE|XXXCC" ]

 

Similarly, there can be many such cs_id's that belong to the same group set hierarchy

 

Once the tas data gets exported from apm_bridge to DX OI, and when you view the groups from within the Services section of DX OI, it must be understood that such similar groups sets as shown above are normalized into one group for multiple cs_ids 421480, 424967, 424968 ingested into DX OI.

 

"uim.groups" :

 

        [ "PRODxxx|SERVABCDE|Xxlow", "PRODxxx|SERVABCDE|XXXCC" ]     : 421480, 424967, 424968 

 

Therefore the page count of 17 x 100 currently seen on your DX OI environment is the accurate value as per the current group count exported.

Some computer system IDs (cs_ids) belong to the same group (uim.groups) - similar groups sets as shown above are normalized into one group for multiple cs_ids 421480, 424967, 424968 ingested into DX OI.

In the latest apm_bridge build, there is a new parameter:

   inventory_alive_time_days

The group data and the group payload data which we retrieve is only for the last two days by default. This parameter value can be increased.

This is useful for exporting more data into DX OI.