How to determine overload condition of the SSO Server?
Article ID: 54746
Updated On:
Products
Issue/Introduction
Description:
While operating CA Single Sign-On (SSO) in a production environment, it is desirable to determine if the SSO Server is performing at its limit or if sufficient processing resources are still available.
This document describes how to evaluate such conditions and indicates possible countermeasures - however no specific tuning advise is given. It is assumed that the reader of this document has a sound understanding of the SSO components available, how they are implemented and in particular has knowledge about the SSO Client / Server connection processing model. If necessary, please read up on these topics in the official SSO Documentation and the SSO r8 Performance Measurement Module (TEC396298)'s documentation.
Solution:
Assuming the SSO Server has been tuned to optimum performance previously, thus t he fork limit value for a particular system should have been determined empirically by simulating a load using the Benchmark Tool and observing the throughput and response times for various values of fork limit.
The ForkLimit value determines the maximum number of worker threads the SSO Server has available for Client connection and other processing activities.
In case of excessive workload, all worker threads become busy and additional requests need to be queued up in the SSO Server's internal Send/Receive Queue. Hence an Administrator wants to be alarmed should such circumstance occur.
This can be visualized by means of the Windows Performance Monitor.
- Prepare the Windows Performance Monitor
SSO Server installation also installs a set of specific counter objects for the Windows Performance Monitor.
In order to utilize these, it is necessary to add them the following way:<Please see attached file for image>
It might be necessary to a djust the graph's Vertical Scale Maximum to accommodate the maximum number of worker threads.<Please see attached file for image>
- Determine if the Worker Threads are all busy and the Server starts queuing additional requests
The graph shows the value for the " Total Threads" (ForkLimit) being constant over time.
In situations were the SSO Server is issued a continuously higher load than it can handle, the "Busy Threads - Total" (green line) accumulate until all free thread resources are finally utilized.
Once the fork limit has been reached, subsequent connection requests are added to the receive queue, until it becomes full - " Waiting requests in queue" (black line).<Please see attached file for image>
This status should be regarded as alarming.
SSO Clients will most likely notice a delay in processing, but will not receive any error as long as queued requests can be processed prior to the ConnectTimeout (to be set in the Client.ini - see below).
Although not highly critical, corrective measurements, such as distributing load over several SSO Servers or using a more powerful machine, should be taken if such situation occurs rather frequently. - Determine if the SSO Server is overloaded and Clients face errors
If the number of "Waiting requests in queue" finally exceed the ReceiveQueueSize, the SSO Server will issue a warning in its log and will reject further connection requests until it recovers.
PSLog.log:
...
_________________________________________________
5/26/2008 1:32:33 PM - WARN (Thread ID: 0x0000301c)
Source : eTrust SSO Server\SSO Server
Message: (0xFF017400) Server is busy (Receive queue is full).
_________________________________________________
... - Reconfigure the SSO Client to avoid timing out
<Please see attached file for image>
Although not optimal, as a short term countermeasure it might be desirable to increase the SSO Client's ConnectTimeout to avoid errors being issued to users.
Client.ini:
...
[NetworkCommunication]
ConnectTimeout=120s
...
Environment
Component:
Resolution
Attachments
Feedback
