Suggestions on what to look for when a policy ran outside of its restrictions/applicability
Article ID: 238752
Updated On:
Products
Issue/Introduction
Sometimes there are situations when something, like a software delivery policy or task, runs out of the scheduled time or targets the wrong set of machines. The customers were concerned that a policy/job/task ran on client machines/servers that shouldn't have.
Tracking this type of issue is very complicated because we don't provide a simple way to identify if it was caused by a glitch or by a human mistake. In most cases we can "assume" what happened but most times any valuable information may be overwritten by other processes.
We are looking for suggestions on what could provide clues to identify what could have caused this issue. Anything else that could help a customer with a root cause analysis?
Environment
ITMS 8.5, 8.6
Resolution
What we usually look at is:
- NS logs around the time the issue happened (under C:\ProgramData\Symantec\SMP\Logs)
- Agent logs around the time the issue happened (under C:\ProgramData\Symantec\Symantec Agent\Logs)
- Check for target(s) definition, possible inclusions/exclusions
- Check for applicability (what is used to verify that the software is needed)
- Check entries under STPatchAssessment.log on a client machine if these are patches (under ...\Program Files\Altiris\Altiris Agent\Agents\SoftwareManagement\Software Delivery\{6D417916-467C-46A7-A870-6D86D9345B61}\cache)
- Enabling auditing with item trackers (https://knowledge.broadcom.com/external/article/173483)
- On one of those policies, right-click on it>Properties, and under the Audit tab should be some sort of entry saying when it was modified
- Server policy XML. For Managed deliveries, if applicability /etc is supposed to be investigated by you, then you will need also detailed export of each component participating in the policy - that will export the entire collection of items associated with software components
- Client-side policy XML (full policies XML under ...\Program Files\Altiris\Altiris Agent\Client Policies) - will be good in case the policy is causing the problem has not disappeared yet from the client.
- Task history folder ...\Program Files\Altiris\Altiris Agent\TaskManagement\TaskHistory from one of the client machines
- For patch policies - ...\Program Files\Altiris\Altiris Agent\Agents\PatchMgmtAgent\InstallLog.csv from one of the client machines
- If the task or policy is no longer present in the SMP Console and you want to see who deleted it, you can use a query like this:
select * from Evt_NS_Item_Management
where ItemGuid = 'AddGUIDhere' and Action = 'Delete'And see what returns. It should return which user deleted the policy. As well, he can use KB 171823 "Find all tables that contain a specific GUID" and look for that GUID in his database.
, - If the issue is no longer happening but wants to be proactive and capture as many log entries as possible, they can increase the size and type of logs that are created:
--On a client machine
1. Please add the following regkeys on your client machines. This should not cause any performance issues. We just need to increase the size and amount of the agent logs and enable the verbose logging on them:
1.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile.
a.) add (if not present already) a New > DWORD (32-Bit) Value and name it 'Severity'.
b.) and give it a hexadecimal value of 'FF'.
2.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile.
a.) add a New > DWORD (32-Bit) Value and name it 'MaxFiles'.
b.) give it a decimal value of 200 (this increases the amount of the log files).
3.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile.
a.) add a New > DWORD (32-Bit) Value and name it 'MaxSize'.
b.) give it a decimal value of 2000 (this increases the size of the agent log files).
4.) Restart the Symantec Management Agent service so the changes can take effect.
5.) After we are done with troubleshooting, please remember to place back the Severity key to a value of 7, and delete the MaxFiles and MaxSize keys to get things back to previous settings.
Basically this should look like this:
[HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Event Logging\LogFile]
"MaxSize"=dword:000007d0
"MaxFiles"=dword:000000c8
"Severity"=dword:000000ff
-----------------------
--On the SMP server:
1. Please check the following regkeys on your SMP server. We just need to increase the size and amount of the NS logs and enable the verbose logging on them:
1.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile.
a.) Check if the regkey 'Severity' exists. If so, check that it has the hexadecimal value of 'FF'
b.) If not add a New > DWORD (32-Bit) Value and name it 'Severity' and give it a hexadecimal value of 'FF'
2.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile.
a.) Check if the regkey 'MaxFiles' exists. If so, check that it has the decimal value of '200'.
b.) If the regkey already exists and has the decimal value of '200', increase it to the decimal value of 300.
c.) If it doesn't exist, add a New > DWORD (32-Bit) Value and name it 'MaxFiles' and give it a decimal value of '300' (this increases the amount of the log files).
3.) Under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile.
a.) Check if the regkey 'MaxSize' exists. If so, check that it has the decimal value of '2000'.
b.) If the regkey already exists and has the decimal value of '2000', increase it to the decimal value of '3000'.
c.) If it doesn't exist, add a New > DWORD (32-Bit) Value and name it 'MaxFiles' and give it a decimal value of '3000' (this increases the size of the log files).
4.) Restart the 'Altiris Services' service so the changes can take effect.
5.) After we are done with troubleshooting, please remember to revert the Severity, MaxFiles, and MaxSize keys to previous value settings.
HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\eXpress\Event Logging\LogFile
"MaxFiles"=dword:0000012c
"MaxSize"=dword:00000bb8
"Severity"=dword:000000ff
Note: Adding extra logging on the SMP Server can add extra CPU and memory usage.
Other things to consider:
Use case:
We had an incident of a Default Policy being on an pushing updates to servers. We need to find ways to prevent it from happening again.
Suggestion:
Review how your company uses the SW Update Plug-in policies. For example, if you leave them off until the patching cycles begin for the month, and if you leave the Default SW Update Plug-in policy (DSUP) off. The DSUP may have been inadvertently enabled and patches installed on a group of servers before their scheduled maintenance period. In this use case, these servers did not reboot, as it was configured to Never.
There is no way to determine who enabled the policy after the issue happened but that in the future, Item Trackers could be used to monitor the DSUP per KB 173483.
You could also enable the DSUP with an installation schedule set far in the future so that Windows System Assessment scans would run, as these will fail if the endpoints are not in the DSUP or a clone of the DSUP.
Additional Information
Feedback
