SIteMinder error in OAUTH flow 'Failed to decrypt'
search cancel

SIteMinder error in OAUTH flow 'Failed to decrypt'

book

Article ID: 209734

calendar_today

Updated On:

Products

SITEMINDER

Issue/Introduction

In OAuth request after entering credentials the below error was reported:

 

FWSTrace.log:

[03/01/2021][06:36:32][28433][139843764152064][][SecureRedirect.java][doInitLog][--------------------------------------------------]
[03/01/2021][06:36:32][28433][139843764152064][][SecureRedirect.java][doInitLog][SAML2 Secure Redirect Service Initialization.]
[03/01/2021][06:36:32][28433][139843764152064][][SecureRedirect.java][doInitLog][--------------------------------------------------]
[03/01/2021][06:36:32][28433][139843764152064][][FWSBase.java][init][Fips140Mode = 1]
[03/01/2021][06:36:32][28433][139843764152064][][SecureRedirect.java][init][SAML2 Secure Redirect Service has been successfully initialized.]
[03/01/2021][06:36:32][28433][139843764152064][][agentcommon][][Requesting data for ConfigManager ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/SmHost.conf and SmAgentConfig ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf]
[03/01/2021][06:36:32][28433][139843764152064][][agentcommon][][Administration Manager is returning data for ConfigManager ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/SmHost.conf and SmAgentConfig ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf]
[03/01/2021][06:36:32][28433][139843764152064][][agentcommon][][Requesting data for ConfigManager ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/SmHost.conf and SmAgentConfig ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf]
[03/01/2021][06:36:32][28433][139843764152064][][agentcommon][][Administration Manager is returning data for ConfigManager ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/SmHost.conf and SmAgentConfig ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][SecureRedirect.java][doGet][SAML2 Secure Redirect Service received GET request.]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][SecureRedirect.java][doGet][Query string is: response_type=code&scope=openid%20consumer&redirect_uri=https://example.com/api/security/oauth/v3/callback&client_id=####&state=###&nonce=&loginpage=lpage&SMPORTALURL=###]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][FWSBase.java][doRequestLog][Requesting Host: XXX.XXX.XXX.XXX Requesting Host IP: XXX.XXX.XXX.XXX Request protocol: HTTP/1.1 Request was secure: true Authentication type: null]
[03/01/2021][06:36:32][28433][139843764152064][][agentcommon][][Requesting data for ConfigManager ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/SmHost.conf and SmAgentConfig ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf]
[03/01/2021][06:36:32][28433][139843764152064][][agentcommon][][Administration Manager is returning data for ConfigManager ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/SmHost.conf and SmAgentConfig ID /opt/CA/secure-proxy/proxy-engine/conf/defaultagent/WebAgent.conf]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][SecureRedirect.java][doGet][Transaction with ID: 48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7 failed. Reason: SERE_GET_EXCEPTION]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][SecureRedirect.java][doGet][Exception caught in class com.netegrity.affiliateminder.webservices.SecureRedirect, method doGet: com.netegrity.siteminder.agentcommon.utils.k: Failed to decrypt.]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][SecureRedirect.java][doGet][Stack Trace: com.netegrity.siteminder.agentcommon.utils.k: Failed to decrypt.
        at com.netegrity.affiliateminder.webservices.f.a(fedfws_obfsc:3997)
        at com.netegrity.affiliateminder.webservices.SecureRedirect.doGet(fedfws_obfsc:189)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at com.netegrity.affiliateminder.webservices.CAFedFilter.doFilter(fedfws_obfsc:58)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:113)
        at com.netegrity.proxy.ProxyValve.processRequest(Unknown Source)
        at com.netegrity.proxy.ProxyValve.invoke(Unknown Source)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.netegrity.siteminder.agentcommon.utils.k: SiteMinder Decryption Exception
        at com.netegrity.siteminder.agentcommon.utils.SmCryptoUtil.c(Unknown Source)
        at com.netegrity.siteminder.agentcommon.utils.SmCryptoUtil.e(Unknown Source)
        at com.netegrity.affiliateminder.webservices.f.a(fedfws_obfsc:3992)
        ... 24 more
Caused by: com.ca.sso.smcrypto.SmCryptoLibException: Digest mismatch.
        at com.ca.sso.smcrypto.bcfipsimpl.SmBaseCrypto.decryptBytes(SmBaseCrypto.java:415)
        ... 27 more

Exception history:
        com.ca.sso.smcrypto.SmCryptoLibException: Digest mismatch.
        com.netegrity.siteminder.agentcommon.utils.k: SiteMinder Decryption Exception
        com.netegrity.siteminder.agentcommon.utils.k: Failed to decrypt.
]
[03/01/2021][06:36:32][28433][139843764152064][48890cfc-660aee71-fe5d93d0-3f56fc65-1d46ce97-7][SecureRedirect.java][doGet][Ending SAML2 Secure Redirect Service request processing with HTTP error 500]

Environment

Release : 12.8

Component : SiteMinder

Cause

HAR file review identified the problem: authentication scheme redirects are messing up the query string.

Following is before being redirected to login page:

https://XXXXXXXXXXXXXXXXXXXXX/affwebservices/secure/secureredirect/consumer/?response_type=code&scope=openid%20consumer&redirect_uri=https://XXXXXXXXXXXXXXXXXXXXX/api/security/oauth/v3/callback&client_id=###&state=###&nonce=&loginpage=lpage&SMPORTALURL=2VzSElxmACbrCcK7vUK9XDcDQvrzVngsRTOZ%2B%2BfYFIM3TtVmtRN0amlOBGaDTaLcA8iA1drAgoHFSOzlndziR6T25TNsMrD9hkZtWQH9hLxAUILI0NN3MCk7zUa790TL

Following is after authentication, redirecting to the same URL:

HTTPS://XXXXXXXXXXXXXXXXXXXXX/affwebservices/secure/secureredirect/consumer/?response_type=code&scope=openid%20consumer&redirect_uri=https://XXXXXXXXXXXXXXXXXXXXX/api/security/oauth/v3/callback&client_id=###&state=###&nonce=&loginpage=lpage&SMPORTALURL=2VzSElxmACbrCcK7vUK9XDcDQvrzVngsRTOZ++fYFIM3TtVmtRN0amlOBGaDTaLcA8iA1drAgoHFSOzlndziR6T25TNsMrD9hkZtWQH9hLxAUILI0NN3MCk7zUa790TL

The URL should be exactly the same before and after the authentication (there can be some additional parameters added or removed but the encoded query string parameters must be preserved, which we don't see here:

SMPORTALURL=2VzSElxmACbrCcK7vUK9XDcDQvrzVngsRTOZ%2B%2BfYFIM3TtVmtRN0amlOBGaDTaLcA8iA1drAgoHFSOzlndziR6T25TNsMrD9hkZtWQH9hLxAUILI0NN3MCk7zUa790TL
SMPORTALURL=2VzSElxmACbrCcK7vUK9XDcDQvrzVngsRTOZ++fYFIM3TtVmtRN0amlOBGaDTaLcA8iA1drAgoHFSOzlndziR6T25TNsMrD9hkZtWQH9hLxAUILI0NN3MCk7zUa790TL

 

Resolution

The customer was using a custom login page ('lpage') instead of the OOTB login.fcc page.
In such situation we advised the customer to use the OOTB login.fcc instead of their 'lpage' (custom login page), which rectified the issue.

After that the customer was able to fix their lpage code so it properly encodes/decodes SMPORTALURL string.

Powered by Wolken Software