SiteMinder policy server generates core dump during shutdown process
Article ID: 216342
Updated On:
Products
Issue/Introduction
--- [ISSUE SUMMARY] ---
SiteMinder policy server generates core dump during shutdown process.
Customer recently upgrade SiteMinder policy server from 12.7 to 12.8.4.
The policy server is running fine most of time, but occasionally there is core dump file generated.
It appears policy server generates core dump during shut down command.
The policy server shutdown was initiated by admin.
pstack
=========
Core was generated by `smpolicysrv'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f3e2855f416 in ?? ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-323.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 libuuid-2.23.2-65.el7_9.1.x86_64 zlib-1.2.7-19.el7_9.x86_64
...
Thread 1 (Thread 0x7f3e21405700 (LWP 126751)):
#0 0x00007f3e2855f416 in ?? ()
No symbol table info available.
#1 0x00007f3e21402e30 in ?? ()
No symbol table info available.
#2 0x0000000000000000 in ?? ()
No symbol table info available.
smexec.log
=============
Received SIGNAL 10
Disabled Service smpolicysrv
Sending SIGTERM to 126684, want first to wait until the child process dies
SIGTERM was sent to 126684, want first to wait until it dies
Deleting pid file ----- /apps/siteminder/ps/config/smpolicysrv.pid
wait interrupted by signal
Enter: HandleSignals(BLOCK_ALL)
Enter: HandleSignals(UNBLOCK_ALL)
Received SIGNAL 10
Disabled Service smservmon
Sending SIGTERM to 126683, want first to wait until the child process dies
SIGTERM was sent to 126683, want first to wait until it dies
Deleting pid file ----- /apps/siteminder/ps/config/smservmon.pid
Going back to the wait function
wait interrupted by signal
Enter: HandleSignals(BLOCK_ALL)
Enter: HandleSignals(UNBLOCK_ALL)
wait failed, errno: 10
smtrace_20210512_094748.637.log.gz
============
[126684][139906861147968][LogMessage:INFO:[sm-Server-02100] Thread 139906861147968 received signal, stopping...][CServer.cpp:6118][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][09:47:27][]
[126684][139906861147968][LogMessage:INFO:[sm-Server-01880] smpolicysrv shutting down][CServer.cpp:4552][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][09:47:27][]
Environment
--- [VERSION & ENVIRONMENT] ---
OS: Red Hat Enterprise Linux Server release 7. 3.10.0-1160.11.1.el7.x86_64 #1 SMP Mon Nov 30 13:05:31 EST 2020 x86_64 x86_64 x86_64 GNU/Linux
Policy server version :12.8 sp4 Build: 2278
Policy Store: IBM LDAP Directory 6.4
ODBC user store Data Sources: Oracle DB
Session store: SQL Server.
Switched from oracle JDK to openJDK 1.8
APM version provided by customer:
Infrastructure Agent: casystemedge-5.9.25-unix-x64.tar.gz
Siteminder agent: 13.2 (caapmsso-13.2.bin)
Java agent: 20.6 (Java-other-20200728_v1.tar)
md5sum ./libIntroscopeNativeDataAPI.so
82cb755b801b7c97f87468cea14e692e ./libIntroscopeNativeDataAPI.so
Cause
Core dump analysis, pointing out "Problem seems like a race condition with the APM Introscope agent during policy server shutdown - likely accessing a pointer that became NULL during shutdown.".
Resolution
Engineering has provided an updated SiteMinder Policy Server installable (Ref Defect: DE504788)
Please open a support ticket to obtain it.
Feedback
