About the Endpoint Protection Error Processor task
search cancel

About the Endpoint Protection Error Processor task

book

Article ID: 151005

calendar_today

Updated On:

Products

Endpoint Protection

Issue/Introduction

 

Resolution

The Symantec Endpoint Protection Error Processor scheduled task runs SymErr.exe /submit in the Endpoint Protection binary folder at midnight –whether the user is logged on or not– or if the system is idle for at least 5 minutes after a logon later in the day. When a process crashes, Windows Error Reporting (WER, using werfault.exe) kicks in and crash-related data is written to %LocalAppData%\Microsoft\Windows\WER\ReportQueue. On systems running Endpoint Protection 14 and higher, most process dumps generated by WER will be redirected to %ProgramData%\Symantec\LocalDumps. SEP's error management handler picks up on any crash and copies the dump to and creates associated data in a SQ_{GUID.EN_US} (SQ being short for SymQual) folder in %ProgramData%\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\ErrMgmt\Queue\Incoming. When SymErr.exe /submit is run, any SQ_{GUID.EN_US} folders present are sent to our fully automated SymQual (Symantec Quality) system for statistical analysis, with the goal of identifying emerging issues.

More details about its operation can be uncovered by following the lifecycle of a crashed process and making a change to the related site-wide policy setting:

  1. Download Bad Application.
  2. Run BadApp.exe, click Crash process and wait until Windows has dealt with the crash (do not hit the Cancel button when notified that "BadApp.exe has stopped working. Windows is checking for a solution to the problem..."). In the background, the following sequence of events unfolds:  
    • Werfault.exe kicks in and %ProgramData%\Microsoft\Windows\WER\Temp\WERxxx.tmp.WERInternalMetadata (containing detailed OS version and process information, as well as application-specific problem signatures, OS-specific dynamic signatures, hardware-specific system information, information about the secure boot state, integrator/dump flags and GUID and creation time of the WER report) and other temporary files are created. While the temporary files created by WER will not be immediately visible in File Explorer, they will be in e.g. Process Monitor.
    • The WER report is saved to %ProgramData%\Microsoft\Windows\WER\ReportQueue\AppCrash_BadApp.exe_{GUID.EN_US}\Report.wer. If an existing crash report is present, it is moved to %ProgramData%\Microsoft\Windows\WER\ReportArchive.
    • After you hit the Close program button when notified that "BadApp.exe has stopped working – A problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available.",  no dump is created in %ProgramData%\Symantec\LocalDumps and no SQ_{GUID.EN_US} folder is created in %ProgramData%\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\ErrMgmt\Queue\Incoming –the reason being that we have not configured it to do so yet, as would normally be the case for most executables on the system after the installation of SEP.
  3. Open Registry Editor (regedit.exe). Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps. In LocalDumps, create new key BadApp.exe, containing the following entries (mirroring similar entries created by SEP):  
    • DumpCount (REG_DWORD, value: 4)
    • DumpFlags (REG_DWORD, value: 0)
    • DumpFolder (REG_EXPAND_SZ, value: C:\ProgramData\Symantec\LocalDumps)
    • DumpType (REG_DWORD, value: 1)

  4. Close Registry Editor. Run BadApp.exe, click Crash process and then the Close program button when prompted. Verify that %ProgramData%\Symantec\LocalDumps\BadApp.exe.<PID>.dmp is created. As per DumpCount registry key, only the last 4 process crash dumps for any given process will be present in the LocalDumps folder. The dump created in the LocalDumps folder is immediately copied to a SQ_{GUID.EN_US} folder in the ErrMgmt\Queue\Incoming folder. Contained in that folder is the following data:

    • BadApp.exe.<PID>.dmp (a process dump, of the type specified by the above mentioned DumpType registry key)
    • SQ_{GUID.EN_US}.plist (a list of all processes active at the time of the dump)
    • SQ_{GUID.EN_US}.dlist (a list of all loaded drivers at the time of the dump, including mapped/image base, image size, flags, load/init order, load count and file name)
    • SQ_{GUID.EN_US}.etl (SEP component tracing at the time of the dump)

  5. Open Task Scheduler (taskschd.msc). Navigate to Task Scheduler Library > Symantec Endpoint Protection. Right-click the Symantec Endpoint Protection Error Processor task and select Run. Witness that all folders in ErrMgmt\Queue\Incoming disappear soon thereafter. The dumps in %ProgramData%\Symantec\LocalDumps remain present.

  6. Crash BadApp.exe again, resulting in a new SQ_{GUID.EN_US} folder in the ErrMgmt\Queue\Incoming folder.

  7. In Symantec Endpoint Protection Manager (SEPM), navigate to Admin > Servers > Local Site. Click Edit Site Properties. In the Data Collection tab, uncheck "Let clients send troubleshooting information to Symantec to resolve product issues faster.". On the client, right-click the SEP tray notification area icon and select Update Policy.

  8. Crash BadApp.exe again and witness that, while a dump is still created in %ProgramData%\Symantec\LocalDumps (the WER redirection of the dump to our folder remains in place), a new SQ_{GUID.EN_US} folder –to which the dump would be copied if the "Let clients send troubleshooting information to Symantec to resolve product issues faster." Local Site property would still be enabled– is no longer created.

  9. Open Task Scheduler (taskschd.msc). Navigate to Task Scheduler Library > Symantec Endpoint Protection. Right-click the Symantec Endpoint Protection Error Processor task. If it is still in a Running state, select End, then right-click it again and select Run. Witness that all folders in ErrMgmt\Queue\Incoming remain and are not processed.

Powered by Wolken Software