Enabling Core files on UNIX with Policy Server

Products

• CA Single Sign On Agents (SiteMinder)
• CA Single Sign On Federation (SiteMinder)
• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• SITEMINDER

Issue/Introduction

--- [ISSUE SUMMARY] ---
SiteMinder policy server generates core dump during shutdown process / startup
Customer recently upgrade to R12.8.2 to R12.8.6a 

There is core dump file generated. 

It appears policy server generates core dump during shut down command / Startup
The policy server shutdown was initiated by admin / SiteMinder user.

Recent changes the user / customer / client is aware of is Red Hat team has updated some regular maintanence patches.

 

Environment

--- [VERSION & ENVIRONMENT] ---

OS: Red Hat Enterprise Linux Server release 7.  3.10.0-1160.11.1.el7.x86_64 #1 SMP Mon Nov 30 13:05:31 EST 2020 x86_64 x86_64 x86_64 GNU/Linux
Policy server version :12.8.6a Build: 2658  
Policy Store: CA Directory 14.1.3

Cause

Core dump analysis, pointing out "Problem seems like a race condition with the APM Introscope agent during policy server Startup / shutdown - likely accessing a pointer that became NULL during shutdown.".

Resolution

Please open a support ticket to obtain it. 

 

Additional Information

How to Enable Core file on UNIX.

How To Enable Core Dumps In RHEL 
1. Edit /etc/security/limits.conf, notice this is set to all processes but if you have a process running as a service account replace the * with the service account (example smpolicysrv). You will have to restart the process for this change to go into effect. 
# vi /etc/security/limits.conf 
#* soft core 0 
* soft core unlimited 

2. Edit /etc/sysctl.conf and add where to put the core file and any naming standard. 
# vi /etc/sysctl.conf 
kernel.core_pattern = /tmp/core-%e-%s-%u-%g-%p-%t 
fs.suid_dumpable = 2 
Filename variables: 
%e is the filename 
%g is the gid (groupID)the process was running under 
%p is the pid of the process 
%s is the signal that caused the dump 
%t is the time the dump occurred 
%u is the uid the process was running under 

3. Edit the /etc/sysconfig/init and set a core file limit. 

# vi /etc/sysconfig/init 
DAEMON_COREFILE_LIMIT=’unlimited’ 

4. Check core status – validates core locations 
# sysctl -p

### Additional insights

-You may need to specify the command 'ulimit -c unlimited' before you invoke the policy server process and reproduce the crash in order to obtain a core file with the crash.

-Also check to be sure you know where the cores will be located via the 'sysctl -p' command.

-You may want to reproduce the issue 3 times so we can be sure all crashes are from the same issue.

-Pkgapp utility should be run against core files in the location the cores where initially generated.(Do not move cores to different directory before running pkgapp)