SiteMinder Web Agent Fails to Initialize Due to Shared Memory Allocation File Exists Error

Products

SITEMINDER CA Single Sign On Agents (SiteMinder) CA Single Sign On Secure Proxy Server (SiteMinder)

Issue/Introduction

The SiteMinder Web Agent for Apache may fail to initialize during service startup, resulting in the agent failing to service requests. The web agent error logs will report failures related to Inter-Process Communication (IPC), specifically regarding shared memory allocation and message bus initialization.

Common Error Messages:

[Error] [CA WebAgent IPC] [...] [CSmSharedSegment::smalloc] Error allocating shared memory segment using key [key] - File exists (17)
[Error] SiteMinder Agent PID Cache error. Failed to initialize PID Cache
[Error] SiteMinder Agent Failed to initialize the message bus.

Environment

Product: SiteMinder Web Agent for Apache
OS: Red Hat Enterprise Linux / Unix-based systems
Components: Apache HTTP Server, LLAWP (Low Level Agent Worker Process)

Cause

This issue is caused by stale or orphaned System V IPC resources (shared memory segments and semaphores) remaining from a previous agent process that did not shut down cleanly.

This typically occurs when using apachectl restart, systemctl restart, or kill -9 commands. These methods often cause the Apache httpd process to restart faster than the Web Agent's LLAWP can release its allocated memory, leading to a race condition where the new agent instance attempts to bind to IPC keys that are still locked by the previous, "zombie" process (1)(2).

Resolution

To resolve this issue, perform a clean shutdown, clear the stale IPC resources, and restart the service using the recommended stop/start sequence.

Prerequisites: Ensure you have administrative (root) access to the server.

Step-by-Step Resolution:

Stop the Apache Web Server: Ensure the service is fully stopped using the standard stop command to allow processes to exit cleanly.

# systemctl stop httpd

Verify Process Termination: Confirm that no Apache or LLAWP processes are lingering.

# ps -ef | grep -i httpd
# ps -ef | grep -i llawp

If processes remain, terminate them manually using kill (use SIGTERM if possible).

Identify and Remove Stale IPC Resources: Use the ipcs command to locate the stale shared memory or semaphore keys referenced in the error logs (e.g., the key [key] mentioned in the error message) (3).

List shared memory:

# ipcs -m | grep -i [key_from_error]

List semaphores:

# ipcs -s | grep -i [key_from_error]

Remove the identified stale resources using ipcrm. Use the numeric ID reported by ipcs (e.g., shmid or semid):

# ipcrm -m [shmid]
# ipcrm -s [semid]

Note: Do not run blanket clear commands (ipcrm -a) on shared hosts, as this may affect other applications.

Start the Apache Web Server: Perform a clean start of the service.

# systemctl start httpd

Verify Initialization: Monitor the Web Agent trace/log file immediately after startup to confirm the PID Cache and message bus initialize without the "File exists" error.

Best Practices to Prevent Recurrence

Avoid 'Restart': Use explicit systemctl stop httpd followed by a pause, then systemctl start httpd. Avoid using restart or graceful reload commands if you encounter frequent IPC collisions.

Instance Identification: If multiple Apache instances or virtual hosts share the same SiteMinder Agent name/host config, ensure they are assigned distinct instance identifiers so their IPC keys do not collide.

Kernel Limits: If the issue persists despite clean shutdowns, check your OS limits (ipcs -l) for SHMMNI, SHMALL, and SEMMNI to ensure the kernel is not exhausting allocated resource limits.