ccSvcHst.exe crashes on 32-bit operating systems after RapidJSON fails to allocate virtual memory
Article ID: 176013
Updated On:
Products
Issue/Introduction
On a 32-bit Windows 7 or 10 system with Symantec Endpoint Protection (SEP) 14.2 RU1 (MP1) ccSvcHst.exe crashes intermittently. In the related ccSvcHst.exe process dumps in %ProgramData%\Symantec\LocalDumps in Windows Debugger, which show the crash occurs as a result of a memory copy operation after many Listener.dll function calls.
Windows Debugger output (read from bottom to top):
STACK_TEXT: 118ef788 6576447b 00000000 05d9b3a8 00000060 msvcr110!memcpy+0x2a 118ef7a4 657734d0 05d9b3a8 00000060 05e4b190 Listener+0x8b 118ef7d4 65790c64 05d9b3a8 00000060 0762ede8 Listener+0x80 118ef834 6579078c 118efa30 118efa40 00000000 Listener+0x2c4 118ef84c 65790f55 118efa30 118efa40 118ef9cc Listener+0x5c 118ef870 657907aa 118efa30 118efa40 118ef8a8 Listener+0xc5 118ef880 65790de1 118efa30 118efa40 118ef9cc Listener+0x7a 118ef8a8 6579079b 118efa30 118efa40 118ef8e0 Listener+0x161 118ef8b8 65790de1 118efa30 118efa40 118ef9cc Listener+0x6b 118ef8e0 6579079b 118efa30 118efa40 118ef918 Listener+0x161 118ef8f0 65790de1 118efa30 118efa40 118ef9cc Listener+0x6b 118ef918 6579079b 118efa30 118efa40 118ef950 Listener+0x161 118ef928 65790de1 118efa30 118efa40 118ef9c0 Listener+0x6b 118ef950 6579079b 118efa30 118efa40 00000000 Listener+0x161 118ef964 65790655 118efa30 118efa40 05b3e371 Listener+0x6b 118ef998 65790223 118ef9c0 118efa30 118efa40 Listener+0xa5 118efa00 6578f058 118efa30 05b3e0f9 7ffc0de8 Listener+0x83 118efa90 6577a5be 118efaf8 118efae0 05b3e1c9 Listener+0xa8 118efb20 6576c75f 7ffc0e98 7ffc0de8 73cbf28e Listener+0x29e 118efb80 71dc818f 09b63f30 71ddf798 71ddba84 Listener+0x20f 118efb9c 71dc82bf 10e001c8 10e036b8 118efbbc ccLib+0x11a 118efbac 71dc8360 09b63f30 10e001c8 118efbf4 ccLib+0x7e 118efbbc 73cbf2e9 09b63f30 98910e46 73cbf28e ccLib+0xc 118efbf4 73cbf2cd 73cbf28e 118efc10 76226359 msvcr110!_beginthreadex+0xb4 118efc00 76226359 10e036b8 76226340 118efc6c msvcr110!_endthreadex+0x102 118efc10 76fa7a94 10e036b8 3415297a 00000000 kernel32!BaseThreadInitThunk+0x19 118efc6c 76fa7a64 ffffffff 76fc8e1b 00000000 ntdll!__RtlUserThreadStart+0x2f 118efc7c 00000000 73cbf28e 10e036b8 00000000 ntdll!_RtlUserThreadStart+0x1b
Environment
- SEP for Windows
Cause
- Listener.dll is part of our Endpoint Detection and Response (EDR) definitions, which are located in %ProgramData%\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\Definitions\EDRDefs. It receives events from BASH's Endpoint Monitor and Query (EMaQ) module, converts them to JSON and sends them to the EDR Store.
- When the ccSvcHst.exe heap is severely fragmented, it will lead to virtual memory address space exhaustion, which in turn may result in a failed memory allocation within the RapidJSON library, causing ccSvcHst.exe to crash.
- RapidJSON is open source. Its allocator class (rapidjson::MemoryPoolAllocator< BaseAllocator >) was implemented without proper exception handling for failed memory allocations. This was reported as RapidJSON issue 1269, but has yet to be resolved.
Resolution
SEP 14.2 RU2 further reduces ccSvchst.exe heap space fragmentation and provides a more permanent solution to this issue. Upgrade to see if this resolves the issue. If it does not, continue below.
Actively monitor and resolve heap space memory fragmentation within ccSvcHst.exe by temporarily disabling Tamper Protection and using Registry Editor to create 32-bit DWORD HKLM\SOFTWARE\(WOW6432Node)\Symantec\Symantec Endpoint Protection\SMC\MemoryMonitor with a value of 1.
If the issue continues to occur, additionally create 32-bit DWORD HKLM\SOFTWARE\(WOW6432Node)\Symantec\Symantec Endpoint Protection\SMC\MemoryMonitorFreq with a value of 1 to 7 (the interval –in hours– with which ccSvcHst.exe will check its heap space memory fragmentation).
Feedback
