Sometimes there are situations when something, like a software delivery policy or task, runs out of the scheduled time or targets the wrong set of machines. You are concerned that a policy/job/task ran on client machines/servers that shouldn't have.
Tracking this type of issue is very complicated because we don't provide a simple way to identify if it was caused by a glitch or by a human mistake. In most cases we can "assume" what happened but most times any valuable information may be overwritten by other processes.
You are looking for suggestions on what could provide clues to identify what could have caused this issue. Anything else that could help with a root cause analysis?
ITMS 8.5, 8.6, 8.7
What we usually look at is:
select * from Evt_NS_Item_Management
where ItemGuid = 'AddGUIDhere' and Action = 'Delete'
And see what returns. It should return which user deleted the policy. As well, he can use KB 171823 "Find all tables that contain a specific GUID" and look for that GUID in his database.
, --On a client machine
1. Please add the following regkeys on your client machines. This should not cause any performance issues. We just need to increase the size and amount of the agent logs and enable the verbose logging on them:
1.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile.
a.) add (if not present already) a New > DWORD (32-Bit) Value and name it 'Severity'.
b.) and give it a hexadecimal value of 'FF'.
2.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile.
a.) add a New > DWORD (32-Bit) Value and name it 'MaxFiles'.
b.) give it a decimal value of 200 (this increases the amount of the log files).
3.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile.
a.) add a New > DWORD (32-Bit) Value and name it 'MaxSize'.
b.) give it a decimal value of 2000 (this increases the size of the agent log files).
4.) Restart the Symantec Management Agent service so the changes can take effect.
5.) After we are done with troubleshooting, please remember to place back the Severity key to a value of 7, and delete the MaxFiles and MaxSize keys to get things back to previous settings.
Basically this should look like this:
[HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile]
"MaxSize"=dword:000007d0
"MaxFiles"=dword:000000c8
"Severity"=dword:000000ff
-----------------------
--On the SMP server:
1. Please check the following regkeys on your SMP server. We just need to increase the size and amount of the NS logs and enable the verbose logging on them:
1.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile.
a.) Check if the regkey 'Severity' exists. If so, check that it has the hexadecimal value of 'FF'
b.) If not add a New > DWORD (32-Bit) Value and name it 'Severity' and give it a hexadecimal value of 'FF'
2.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile.
a.) Check if the regkey 'MaxFiles' exists. If so, check that it has the decimal value of '200'.
b.) If the regkey already exists and has the decimal value of '200', increase it to the decimal value of 300.
c.) If it doesn't exist, add a New > DWORD (32-Bit) Value and name it 'MaxFiles' and give it a decimal value of '300' (this increases the amount of the log files).
3.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile.
a.) Check if the regkey 'MaxSize' exists. If so, check that it has the decimal value of '2000'.
b.) If the regkey already exists and has the decimal value of '2000', increase it to the decimal value of '3000'.
c.) If it doesn't exist, add a New > DWORD (32-Bit) Value and name it 'MaxFiles' and give it a decimal value of '3000' (this increases the size of the log files).
4.) Restart the 'Altiris Services' service so the changes can take effect.
5.) After we are done with troubleshooting, please remember to revert the Severity, MaxFiles, and MaxSize keys to previous value settings.
HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile
"MaxFiles"=dword:0000012c
"MaxSize"=dword:00000bb8
"Severity"=dword:000000ff
Note: Adding extra logging on the SMP Server can add extra CPU and memory usage.
Other things to consider:
Use case:
We had an incident of a Default Policy being on an pushing updates to servers. We need to find ways to prevent it from happening again.
Suggestion:
Review how your company uses the SW Update Plug-in policies. For example, if you leave them off until the patching cycles begin for the month, and if you leave the Default SW Update Plug-in policy (DSUP) off. The DSUP may have been inadvertently enabled and patches installed on a group of servers before their scheduled maintenance period. In this use case, these servers did not reboot, as it was configured to Never.
There is no way to determine who enabled the policy after the issue happened but that in the future, Item Trackers could be used to monitor the DSUP per KB 173483.
You could also enable the DSUP with an installation schedule set far in the future so that Windows System Assessment scans would run, as these will fail if the endpoints are not in the DSUP or a clone of the DSUP.