Getting multiple entries for "Initiating safe system restart" on the Agent logs

book

Article ID: 211292

calendar_today

Updated On:

Products

Management Platform (Formerly known as Notification Server)

Issue/Introduction

The customer reported that after their recent patching cycle, there was a machine that keeps adding entries in the Agent logs about system restart, even after the machine has been rebooted:

Initiating safe system restart
-----------------------------------------------------------------------------------------------------
Process: AeXNSAgent.exe (2876), Thread ID: 1280, Module: AeXNSAgent.exe
Priority: 4, Source: PowerAction

 

The System Event logs shows something like this:

The process C:\Program Files\Altiris\Altiris Agent\AeXNSAgent.exe (ClientMachine01) has initiated the restart of computer ClientMachine01 on behalf of user NT AUTHORITY\SYSTEM for the following reason: Application: Maintenance (Planned)
 Reason Code: 0x80040001
 Shutdown Type: restart
 Comment: Shutdown requested by Symantec Management Agent

 

This is an example of what was noticed on the Agent logs:

1. Below is the very first record about a power action 

Pending power action '0x00010001: reboot, maintenance' scheduled for 2021-03-10 13:55:15 -7:00 UTC
-----------------------------------------------------------------------------------------------------
Date: 3/10/2021 6:54:15 AM, Tick Count: 136203 (00:02:16.2030000), Size: 329 B
Process: AeXNSAgent.exe (2876), Thread ID: 1280, Module: AeXNSAgent.exe
Priority: 4, Source: PowerAction

It indicates that the previous reboot command was presented in the registry SOFTWARE\Altiris\Altiris Agent\Dynamic Data\PowerActions during client start.

2. Then Symantec Management Agent (SMA) starts, detects that restart is needed and is trying to reboot machine at 6:55:15, 1:13 after the start. 

Starting the scheduled power action '0x00010001: reboot, maintenance' now
-----------------------------------------------------------------------------------------------------
Date: 3/10/2021 6:55:15 AM, Tick Count: 196203 (00:03:16.2030000), Size: 304 B
Process: AeXNSAgent.exe (2876), Thread ID: 1280, Module: AeXNSAgent.exe
Priority: 4, Source: PowerAction

Initiating safe system restart
-----------------------------------------------------------------------------------------------------
Date: 3/10/2021 6:55:15 AM, Tick Count: 196203 (00:03:16.2030000), Size: 261 B
Process: AeXNSAgent.exe (2876), Thread ID: 1280, Module: AeXNSAgent.exe
Priority: 4, Source: PowerAction

3. SMA fails to terminate gracefully because the Patch plugin hung during shutdown, so Symantec Management Agent (SMA) kills itself after 12 seconds.

Agent Service will be terminated due to an excess of plugin stopping time over the limit or unexpected stopping failure!
-----------------------------------------------------------------------------------------------------
Date: 3/10/2021 6:55:28 AM, Tick Count: 209375 (00:03:29.3750000), Size: 353 B
Process: AeXNSAgent.exe (2876), Thread ID: 2424, Module: AeXNSAgent.exe
Priority: 1, Source: Altiris Agent

4. The reboot should be occurring at this point, the service starts presumable after the reboot at  7:00:54, more than 5 minutes after the reboot. This looks OK, because the delayed services start about 3 minutes after Windows boot 

Started: '"C:\Program Files\Altiris\Altiris Agent\AeXNSAgent.exe"' 
PID: 4320 
Parent: '\Device\HarddiskVolume1\Windows\System32\services.exe' 
Parent PID: 680
-----------------------------------------------------------------------------------------------------
Date: 3/10/2021 7:00:54 AM, Tick Count: 134671 (00:02:14.6710000), Size: 396 B
Process: AeXNSAgent.exe (4320), Thread ID: 5528, Module: AeXNSAgent.exe
Priority: 8, Source: ProcessStartup

5. Then SMA is checking if there are any pending restart commands in registry and finds the same command again! Look at the first mention under entry 1 above and this one:

Pending power action '0x00010001: reboot, maintenance' scheduled for 2021-03-10 14:01:55 -7:00 UTC
-----------------------------------------------------------------------------------------------------
Date: 3/10/2021 7:00:55 AM, Tick Count: 136078 (00:02:16.0780000), Size: 329 B
Process: AeXNSAgent.exe (4320), Thread ID: 4948, Module: AeXNSAgent.exe
Priority: 4, Source: PowerAction

Cause

In this particular scenario, when the system needed to be rebooted, the end-user stopped/paused/cancelled the reboot.

The Altiris Agent  (Symantec Management Agent or SMA) reads the instruction found under SOFTWARE\Altiris\Altiris Agent\Dynamic Data\PowerActions during client start. This system had the following:
[HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Dynamic Data\PowerActions]
"ScheduledPowerAction"=dword:00010001

This value is meant to be volatile and cleared out by the OS after a reboot. If the registry key under “SOFTWARE\Altiris\Altiris Agent\Dynamic Data” is still there, then it is NOT volatile and it is causing the Altiris Agent to consistently think that there is still an scheduled reboot. 
How it works is agent creates the volatile key “Dynamic Data” and saves the pending reboot commands in there. The volatile keys are not stored in registry files, they cannot survive the reboot, all the volatile keys are missing after the reboot. So if this key is not volatile then it will survive the reboot and will still contain the reboot command when SMA starts.
This key is managed by the OS. SMA never cleans it, that’s the whole point. SMA creates the key as volatile and writes the reboot command in there. The command will be there until the actual reboot takes place.  If you restart SMA without restarting the machine SMA will read the reboot command on every start.  The whole registry key disappears after the reboot even before SMA starts.

Environment

ITMS 8.5

Resolution

1. Reboot machine. 

2. Check right before the Altiris Agent service starts if this "ScheduledPowerAction"=dword:00010001" regkey still exists under [HKEY_LOCAL_MACHINE\SOFTWARE\Altiris\Altiris Agent\Dynamic Data\PowerActions]

3. If so, delete "ScheduledPowerAction" regkey and restart Altiris Agent service.