Policy Server doesn't connect back to the first Session Store
book
Article ID: 130316
calendar_today
Updated On:
Products
CA Single Sign On Secure Proxy Server (SiteMinder)AXIOMATICS POLICY SERVERCA Single Sign On SOA Security Manager (SiteMinder)CA Single Sign-On
Issue/Introduction
We're running Policy Server and when we have several ODBC Session Stores, we see the ODBC connection unequally distributed among the ODBC instances. How can we get the connections distributed equally ?
Environment
Policy Server 12.52SP1CR02 on RedHat 5; JDK 1.7.0_141 32bit;
2 Session Store on ODBC Oracle;
Resolution
Even if the ODBC driver allows the usage of alternateservers and loadbalancing features, these will proceed only with failover, even if the feature is called loadbalancing. The value of loadbalancing determine the order in which the failover occur on the configuration (in serie or random) :
Configure an Oracle Data Source for CA Single Sign-On
AlternateServers=
If the primary server is not accepting connections, specifies the connection failover to the other Oracle nodes.
Turns on client load balancing, which helps to distribute new connections to keep RAC nodes from being overwhelmed with connection requests. When enabled, the order in which primary and alternate database servers are accessed is random.
Inconsistent behavior from PS to DB Loadbalancing https://comm.support.ca.com/kb/inconsistent-behavior-from-ps-to-db-loadbalancing/kb000119929
In order to avoid trailing connections to down ODBC instances, in system_odbc.ini, configure the following :
- Set the KeepAlive parameter to 1 in order to close any connection to a down server;
CA SSO Oracle Server Wire Protocol
KeepAlive=0 NOTE: The attributes in the example (above) reflect the hardcoded, default values. Adding the attribute and modifying the value will override the default value with the user-designated value.
Specifies the number of seconds to keep inactive connections open in a connection pool. An inactive connection is a database session that is not associated with an ODBC connection handle, that is, a connection in the pool that is not in use by an application.
This will avoid the connections to be kept open to ODBC instances which are down.
More, you can also configure 2 separated data source and remove the configuration of alternateservers and loadbalancing from the driver configuration will make all the odbc connection going back to the Primary Session Store. Note that from documentation, we support failover only, and the driver loadbalancing configuration stands for failover too.
Here we've removed the alternateservers and LoadBalancing configuration and we've defined 2 separate Session Store data sources, 1 for the Primary and another one for the Secondary.
In the SmConsole, we've configured the Session Store connection putting each of the datasources, separated with a coma "," :
sm.registry :
HKEY_LOCAL_MACHINE\SOFTWARE\Netegrity\SiteMinder\CurrentVersion\Database\SessionServer=235352101 Data Source=Session Primary, Session Secondary; REG_SZ Enabled= 0x1; REG_DWORD MaxConnections= 0x20; REG_DWORD Password= {RC2}cJhuVJ9LzQf12y9hk6FdGc9ztDo+Y8Em; REG_SZ ProviderNamespace= ODBC:; REG_SZ Use Default= 0; REG_DWORD User Name= SMUSER; REG_SZ
Once done, we've started Policy Server, and we've seen it using the Primary Session Store. When the Primary Session Store went off line, Policy Server moved connections to the Secondary Session Store. When the Primary Session Store came back online, then Policy Server moved back the connections to the Primary Session Store :
smps.log :
[13473/4151690960][Fri Dec 28 2018 03:08:08][CServer.cpp:4951][INFO][sm-Server-01430] Logging and Trace output in local time. TimeZone: [GMT-5:00]. Daylight Savings: 0
[13473/3799419760][Fri Dec 28 2018 03:08:12][CSmDbSessionManager.cpp:585][INFO][sm-Server-04350] Using ODBC 'Session Data' data source 'Session Primary'.
[13473/4151690960][Fri Dec 28 2018 03:08:12][SmSSInDBStore.cpp:299][INFO][sm_LoginLogout_02014] Initialized Session Server. Schema version is 3, lVersion
[13473/3715414896][Fri Dec 28 2018 03:09:12][CSmDbSessionManager.cpp:180][INFO][sm-Server-04350] Using ODBC 'Session Data' data source 'Session Primary'.
[13473/4065667952][Fri Dec 28 2018 03:17:38][CSmDbSessionManager.cpp:147][INFO][sm-Server-04330] Failing over to 'Session Data' data source 'Session Secondary'.
[13473/3715414896][Fri Dec 28 2018 03:22:12][CSmDbSessionManager.cpp:155][INFO][sm-Server-04340] Failing back to 'Session Data' data source 'Session Primary'.
Netstat output during the use case reproduction :
We start the Policy Server :
Fri Dec 28 03:08:13 EST 2018
tcp 0 0 10.162.31.240:11821 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:44748 10.162.31.240:1521 ESTABLISHED 12171/xe_pmon_XE
tcp 0 0 10.162.31.240:31701 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
We put load and Policy Servers build connections to the Primary Session Store (.240):
Fri Dec 28 03:16:07 EST 2018
tcp 0 0 10.162.31.240:11821 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:59662 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:32757 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:44748 10.162.31.240:1521 ESTABLISHED 12171/xe_pmon_XE
tcp 0 0 10.162.31.240:20816 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:22829 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:31004 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 689 10.162.31.240:31701 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:65377 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
We shut down the Primary Session Server (.240) :
Fri Dec 28 03:17:37 EST 2018
tcp 0 0 10.162.31.240:11821 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
Policy Server fails over connections to the Secondary Session Store (.106) :
Fri Dec 28 03:18:38 EST 2018
tcp 0 0 10.162.31.240:11821 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:17993 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:55709 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:36960 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:51719 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:22171 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:58619 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:48457 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
We start over the Primary Session Server :
Fri Dec 28 03:21:32 EST 2018
tcp 0 0 10.162.31.240:11821 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:45040 10.162.31.240:1521 ESTABLISHED 18172/xe_pmon_XE
tcp 0 0 10.162.31.240:17993 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:55709 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:36960 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:51719 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:22171 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:58619 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:48457 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
Policy Server notices it and starts to move back all connections to the Primary Session Store :
Fri Dec 28 03:21:41 EST 2018
tcp 0 0 10.162.31.240:11821 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:45040 10.162.31.240:1521 ESTABLISHED 18172/xe_pmon_XE
tcp 0 0 10.162.31.240:39907 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:17993 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:55709 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:55979 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:36960 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:51719 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:22171 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:27555 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:58619 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:48457 10.162.31.106:1521 ESTABLISHED 13473/smpolicysrv
Until all connections are back to the Primary Session Store :
Fri Dec 28 03:21:48 EST 2018
tcp 0 0 10.162.31.240:45040 10.162.31.240:1521 ESTABLISHED 18172/xe_pmon_XE
tcp 0 0 10.162.31.240:39907 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:14352 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:36902 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 679 10.162.31.240:55979 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:27776 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:51390 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:27555 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
tcp 0 0 10.162.31.240:31303 10.162.31.240:1521 ESTABLISHED 13473/smpolicysrv
As stated earlier in this case, you won't be able to achieve loadbalancing between the Session Stores. So you will never get connections equally distributed between nodes.
As our documentation states, Session Stores instances should be set as failover.
Policy Server to Session Store Communication
Allows for failover. If a primary session store fails, Policy Servers failover to a secondary session store.