Enable monitoring for configuration changes with the audit probe
Article ID: 128820
Updated On:
Products
Issue/Introduction
Customer needs to know how to configure the audit probe to track configuration changes by users and settings
Environment
- UIM 20.x
- audit v9.04 or later
Cause
Resolution
The audit probe does not capture all types of user interactions. The audit probe has restrictions to work with IM, and it will not record who made the change if changes are made in Admin Console or MCS. Furthermore, each probe has different capabilities in terms of what can be captured. Also, if you need to track user's when they make changes to probe configuration's, you have to tell it what user to track the configuration changes for when you setup audit rules/profiles.
Relevant audit tables include:
- AUDIT_EVENT
- AUDIT_CONFIG_CHANGE
By default, the probe will only keep 30 days of audit records in these tables. This is configurable from the probe's Configure GUI.
Note that any changes made to a probe configuration through the monitoring tab of the UMP USM portlet, Admin Console or MCS, will not be recorded by the audit probe.
All of the most granular information is stored in the AUDIT_EVENT table and the audit.log.
select * from audit_event
Sample rows from the audit_event table:
event_id event_type event_time event_descr origin domain hub robot probe user_name user_cmd user_ip status config_changes checkpoint_id
1 1 2019-03-04 12:26:31.000 Login: succeeded for audit abcxx01_hub abcxx01_domain abcxx01_hub magka04-u157656 hub NULL login 10.162.x.xxx 0 0 0
2 1 2019-03-04 12:26:50.000 Turn on audit abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 controller administrator _audit_type 10.162.x.xxx 0 0 0
3 0 2019-03-04 12:26:51.000 Audit initiated abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 controller NULL NULL NULL 0 0 1
4 0 2019-03-04 12:26:51.000 Audit initiated abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 spooler NULL NULL NULL 0 0 1
5 0 2019-03-04 12:26:51.000 Audit initiated abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 hdb NULL NULL NULL 0 0 1
6 0 2019-03-04 12:26:51.000 Audit initiated abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 cdm NULL NULL NULL 0 0 1
7 0 2019-03-04 12:26:51.000 Audit initiated abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 wasp NULL NULL NULL 0 0 1
8 0 2019-03-04 12:26:51.000 Audit initiated abcxx01_hub abcxx01_domain abcxx01_hub xyzxx01 xendesktop NULL NULL NULL 0 0 1
select * from audit_event where event_id IN (select event_id from audit_config_change where new_value is NOT NULL)
event_id event_type event_time event_descr origin domain hub robot probe user_name user_cmd user_ip status config_changes checkpoint_id
227 1 2019-03-05 11:33:55.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 processes administrator probe_config_set 10.162.x.xxx 0 1 6
228 1 2019-03-05 11:43:45.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 audit administrator probe_config_set 10.162.x.xxx 0 1 6
229 1 2019-03-05 11:44:08.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 processes administrator probe_config_set 10.162.x.xxx 0 1 7
241 1 2019-03-05 11:48:50.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 processes audit_test probe_config_set 10.162.x.xxx 0 1 8
242 1 2019-03-05 11:49:48.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 processes audit_test probe_config_set 10.162.x.xxx 0 1 9
244 1 2019-03-05 11:50:09.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 processes audit_test probe_config_set 10.162.x.xxx 0 1 10
245 1 2019-03-05 11:51:57.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 audit audit_test probe_config_set 10.162.x.xxx 0 1 7
246 1 2019-03-05 11:52:31.000 NULL xyzxx01_hub xyzxx01_domain xyzxx01_hub xyzxx01 processes audit_test probe_config_set 10.162.x.xxx 0 1 11
You can experiment with the query to achieve what you want.
When you configure the threshold, you have to pick a column and character type from the results of the query run, and make sure you set the Operator, e.g., =
Options:
1. Use the sql_response probe to run a query such as:
select * from audit_event where...
or
select <attrib1,attrib2,attrib3> from audit_event where
or something like:
select * from audit_event where event_id IN (select event_id from audit_config_change where variable = 'config_changes')
and use the Value Tab/function to alarm based on the output.
or use the sqlserver probe and create a custom checkpoint.
Important Notes:
With hub 'Override: audit on' selected in the hub GUI and audit probe log set to loglevel 5, as soon as you make a change to a probe, e.g., the cdm probe on a robot, you should see it in the log. In this scenario, no audit key value was set at the robot level (controller):
Mar 22 13:12:04:059 [5092] audit: [0084] hub=xxxxxxx-E19931_hub robot=xxxxxxx-E19931 probe=cdm config_change=yes
Mar 22 13:12:04:059 [5092] audit: PreProcessArg - /disk/alarm/fixed/E:\,
Mar 22 13:12:04:059 [5092] audit: PreProcessArg - active,
Mar 22 13:12:04:059 [5092] audit: PreProcessArg - no,
Mar 22 13:12:04:059 [5092] audit: PreProcessArg - yes,
Mar 22 13:12:04:063 [5092] audit: [0084] change: section=/disk/alarm/fixed/E:\ key=active no ->yes
Mar 22 13:12:04:063 [5092] audit: PreProcessArg - /disk/alarm/fixed/C:\,
Mar 22 13:12:04:063 [5092] audit: PreProcessArg - active,
Mar 22 13:12:04:063 [5092] audit: PreProcessArg - no,
Mar 22 13:12:04:063 [5092] audit: PreProcessArg - yes,
Mar 22 13:12:04:068 [5092] audit: [0084] change: section=/disk/alarm/fixed/C:\ key=active no ->yes
Mar 22 13:12:04:068 [5092] audit: AnalyzeAuditRecord leave
Mar 22 13:12:04:068 [5092] audit: SREPLY: status = 0(OK) ->10.162.xx.xx/48002
Mar 22 13:12:04:068 [5092] audit: DataCallback leave
As you can tell from the audit.log, it doesn't show which user made the change when you look at the audit log at loglevel 5, although the change and activity shows up in the audit probe GUI itself.
When you configure the hub to allow the robot to set the audit level, e.g., audit = 8, set in the robot (controller) probe via Raw Configure, you can see the configuration change by the user, in this case 'administrator,' in the audit.log as long as you create a NEW filter in the audit probe GUI.
Mar 22 13:18:36:011 [6124] audit: data[4] PDS_PPDS 333
Mar 22 13:18:36:011 [6124] audit: event_id PDS_I 3 84
Mar 22 13:18:36:011 [6124] audit: event_type PDS_I 2 1
Mar 22 13:18:36:011 [6124] audit: event_time PDS_I 11 1553274723
Mar 22 13:18:36:011 [6124] audit: origin PDS_PCH 19 xxxxxxx-E19931_hub
Mar 22 13:18:36:011 [6124] audit: domain PDS_PCH 22 xxxxxxx-E19931_domain
Mar 22 13:18:36:011 [6124] audit: hub PDS_PCH 19 xxxxxxx-E19931_hub
Mar 22 13:18:36:011 [6124] audit: robot PDS_PCH 15 xxxxxxx-E19931
Mar 22 13:18:36:011 [6124] audit: probe PDS_PCH 4 cdm
Mar 22 13:18:36:011 [6124] audit: user_name PDS_PCH 14 administrator
Mar 22 13:18:36:011 [6124] audit: user_cmd PDS_PCH 17 probe_config_set
Mar 22 13:18:36:011 [6124] audit: user_ip PDS_PCH 13 10.162.xx.xx
Mar 22 13:18:36:011 [6124] audit: status PDS_I 2 0
Mar 22 13:18:36:011 [6124] audit: config_changes PDS_I 2 1
IMPORTANT!!! If you do NOT setup a specific filter in the audit probe GUI you will NOT see the user_name entries showing who made the change, in the audit.log
You can add a filter and test it and you will see log entries such as the ones above. Then you can delete the filter and retest it and youll notice that user_name will no longer be included in the audit log.
Here is an example of what fields were used for tracking changes by a user to the cdm probe:
2. It may be advantageous to use the sqlserver probe and create a custom checkpoint because you get a bit more flexibility in the QOS and alarm threshold configuration/setup.
https://comm.support.ca.com/kb/how-to-create-a-new-custom-checkpoint-using-the-sqlserver-probe/kb000039422
Additional Information
You can use the sql_response probe to run a query on the audit tables to pull out the information and place it into an alarm.
SELECT ae.event_id, event_time, user_ip, robot,probe,USER_NAME,user_cmd,acc.operation,acc.section,acc.variable,acc.old_value,acc.new_value
FROM [CA_UIM].[dbo].[AUDIT_EVENT] ae
join [CA_UIM].[dbo].[AUDIT_config_change] acc on ae.event_id = acc.event_id
where ae.probe = 'cdm'
and ae.user_cmd = 'probe_config_set'
and event_time > dateadd(minute, -5, getdate())
so it looks for events in the past 5 minutes and therefore the probe must run every 5 mins.
So every time you edit the cdm probe you get an alarm, then you can use that alarm to send an email.
<check_for_audit_rows>
active = yes
description =
connection = ca_uim_database
interval = 30sec
timeout = 30sec
query =
file = check_for_audit_rows.sql
query_use = file
cursor = client
timeout_error = major
no_record_severity = inactive
no_record_message = $profile/$server/$database - query returned no record
no_record_i18n_token = as#database.sql_response.no_record
no_record_supp_key = value
no_record_qos =
scheduling = rules
source =
<response>
active = no
qos = no
network = inclusive
type = prepare
pings = 0
low_severity = warning
low_message = $profile/$server/$database - response time [$time ms] exceeded threshold [$threshold]
low_threshold =
low_i18n_token =
high_severity = major
high_message = $profile/$server/$database - response time [$time ms] exceeded threshold [$threshold]
high_threshold =
high_i18n_token =
clear_severity = clear
clear_message = $profile/$server/$database - alarm cleared: time = $time ms
clear_i18n_token =
</response>
<count>
active = no
qos = no
low_severity = warning
low_message = $profile/$server/$database: $rows rows in select is $condition threshold $threshold
low_threshold =
low_i18n_token =
high_severity = major
high_message = $profile/$server/$database: $rows rows in select is $condition threshold $threshold
high_i18n_token =
high_threshold =
clear_severity = clear
clear_message = $profile/$server/$database - alarm cleared: rows = $rows
clear_i18n_token =
condition = >
</count>
<value>
active = yes
qos = no
NULLs_handling = as_zero
row_key = event_time
<alarm_lists>
<1>
name = alarm_message
columns = user_cmd
column_type =
low_severity = inactive
low_message = $profile/$server/$database: value $value is $condition threshold $threshold
low_threshold =
high_severity = major
high_message = $profile - $value $USER_NAME $operation, $section , $variable, $old_value, $new_value
high_threshold = probe_config_set
clear_severity = clear
clear_message = $profile/$server/$database - alarm cleared, value = $value
low_condition = =
comparison = character
</1>
</alarm_lists>
</value>
</check_for_audit_rows>
Attachments
Feedback
