CA Single Sign-On SAP WebAS ERP agent fails to initialize

book

Article ID: 41006

calendar_today

Updated On:

Products

CA Single Sign On Secure Proxy Server (SiteMinder) CA Single Sign On SOA Security Manager (SiteMinder) CA Single Sign-On

Issue/Introduction

Issue:

The 12.0 and 12.51 SAP WebAS ERP agent may not initialize correctly upon startup, due a timeout in the trusted host handshake with the policy server which defaults to 10 seconds.  The error messages logged in the SAP WebAS default.trc and/or security.trc log files as SAP WebAS starts are:

“Return code from init():  -1”

“Agent initialization failed...Check agent name, shared secret and FIPS mode compatibility with Policy Server. Also verify that the specified Policy Servers are reachable”

No error messages are presented in the policy server trace log, or smps.log files indicating that an agent handshake failed.  If a packet capture is taken of traffic between the agent and policy server systems while SAP WebAS is started, the policy server sends a RST packet to the agent exactly 10 seconds after the agent sent the first SYN packet.

Environment:  

All supported versions and platforms of the CA Single Sign-On policy server, and all supported platforms for the 12.0 and 12.51 SAP WebAS ERP agent as covered in the Platform Support Matrices at the following locations:

12.0: https://support.ca.com/phpdocs/7/5262/5262_session_ERP_system.pdf

12.51: https://support.ca.com/phpdocs/7/5262/5262_CA_Single_Sign_On_Agent_12_51_For_ERP_PSM.pdf

Cause: 

The policy server has a timeout of 10 seconds to allow an agent to complete the handshake process of connecting to the policy server and presenting is trusted host shared secret data.  Due to potential network delays and the load within the SAP WebAS application’s JVM as the application server starts, the agent’s initial handshake with the policy server may exceed this 10 second limit and cause the agent to not function.

Resolution:

The 12.51 CR6 and 12.52 SP1 CR2 releases of the CA Single Sign-On policy server introduce a new registry setting “AcceptTimeout” which enables an administrator to configure a longer timeout for agent handshakes, allowing the SAP WebAS ERP agent to initialize successfully. Upgrade to the 12.51 CR6 or newer, or 12.52 SP1 CR2 or newer policy server and follow the steps below based on the operating system the policy server runs on:

Windows 64 bit:

Use regedit and locate the “HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Netegrity\SiteMinder\CurrentVersion\PolicyServer” branch of the registry and add a new DWORD value “AcceptTimeout” and set it to a decimal of the desired number of seconds, such as 60.

Windows 32 bit:

Use regedit and locate the “HKEY_LOCAL_MACHINE\SOFTWARE\ \Netegrity\SiteMinder\CurrentVersion\PolicyServer” branch of the registry and add a new DWORD value “AcceptTimeout” and set it to a decimal of the desired number of seconds, such as 60.

Solaris and Linux (any bit level):

Locate the “<policy server install>/registry/sm.registry” file, and navigate to the section of the file for the “HKEY_LOCAL_MACHINE\SOFTWARE\ \Netegrity\SiteMinder\CurrentVersion\PolicyServer” branch. Add a new DWORD value “AcceptTimeout” and set it to the hexadecimal value for the desired number of seconds, such as 0x3c for 60 seconds.

Validate the setting was applied correctly (applies to all listed operating systems):

 

Restart the policy server process, run the “smpolicysrv –publish” command and then validate the ACCEPT_TIMEOUT setting published in the file “<policy server install>/log/smpublish.XML” reflects the correct number of seconds for your changes, not the default 10 seconds.

Environment

Release:
Component: SMSSW