Advanced Auth and SiteMinder Integration Fails

Products

• SITEMINDER
• CA Single Sign On Federation (SiteMinder)
• CA Single Sign On Secure Proxy Server (SiteMinder)

Issue/Introduction

Policy server crashes during Arcot authentication.  This is a newer upgraded integration from existing setup.

Policy server is upgraded to SiteMinder 12.8.0600.2649. 

CA Arcot Adapter Version is 9.0.0 or 9.1.1

Custom authentication scheme is using lib ArcotSiteMinderAdapter.

smtracedefault.log

[07/15/2022][16:24:04][16:24:04.481][][][][][][8096][10700][][][][][][][][][][SmAuthUser.cpp:959][AuthenticateDsUser][][][][][][][][][][][][][][][][][][][0][][][][][][][][Leave function AuthenticateDsUser][][][][00:00:00.000000][][][][][]
[07/15/2022][16:24:04][16:24:04.481][][][][][][8096][10700][][][][][][][][][][SmAuthHtml.cpp:559][SmAuthenticate][][][][][][][][][][][][][][][][][][][2][][][][][][][][Leave function SmAuthenticate][][][][00:00:00.000000][][][][][]
[07/15/2022][16:24:04][16:24:04.481][][][][][][8096][10700][][][][][][][][][][SmAuthUser.cpp:833][SetDsUserProp][][][][][][][][][][][][][][][][][][][][][][][][][][][Enter function SetDsUserProp][][][][][][][][][]
[07/15/2022][16:24:04][16:24:04.481][][][][][][8096][10700][][][][][][][][][][SmAuthUser.cpp:3458][CSmAuthUser::SetPropIndex][][][][][][][][][][][][][][][][][][][][][][][][][][][Enter function CSmAuthUser::SetPropIndex][][][][][][][][][]
[07/15/2022][16:24:04][16:24:04.481][][][][][][8096][10700][][][][][][][][][][SmAuthUser.cpp:3605][CSmAuthUser::SetPropIndex][][][][][][][][][][][][][][][][][][][1][][][][][][][][Leave function CSmAuthUser::SetPropIndex][][][][00:00:00.000000][][][][][]
[07/15/2022][16:24:04][16:24:04.481][][][][][][8096][10700][][][][][][][][][][SmAuthUser.cpp:856][SetDsUserProp][][][][][][][][][][][][][][][][][][][0][][][][][][][][Leave function SetDsUserProp][][][][00:00:00.000000][][][][][]

smps.crash.pstack_4200__20220721_162114.txt

strstr()-[0x7FF9E3B21DC0] - C:\Windows\SYSTEM32\VCRUNTIME140.dll
SmAuthenticate()-[0x7FF9C1F68BE0] - D:\CA\siteminder\bin\smauthhtml.DLL
CertStore::save()-[0x7FF9C3F265E0] - D:\CA\siteminder\bin\ArcotSiteMinderAdapter.dll
CertStore::save()-[0x7FF9C3F265E0] - D:\CA\siteminder\bin\ArcotSiteMinderAdapter.dll
CertStore::save()-[0x7FF9C3F265E0] - D:\CA\siteminder\bin\ArcotSiteMinderAdapter.dll
CertStore::save()-[0x7FF9C3F265E0] - D:\CA\siteminder\bin\ArcotSiteMinderAdapter.dll
SmAuthenticate()-[0x7FF9C3F21F20] - D:\CA\siteminder\bin\ArcotSiteMinderAdapter.dll
CSmAuthUser::Authenticate()-[0x7FF9DC63B420] - D:\CA\siteminder\bin\SmAuth.dll
CSmAuthUser::VerifyUserCredentials()-[0x7FF9DC63AEB0] - D:\CA\siteminder\bin\SmAuth.dll
CSmAuthUser::AuthenticateUserDir()-[0x7FF9DC6362E0] - D:\CA\siteminder\bin\SmAuth.dll
CSmSessionAssuranceCache::~CSmSessionAssuranceCache()-[0x7FF6DEE06610] - D:\CA\siteminder\bin\smpolicysrv.exe
CSmAuthSchemeCache::~CSmAuthSchemeCache()-[0x7FF6DEDD0530] - D:\CA\siteminder\bin\smpolicysrv.exe
CSmSessionAssuranceCache::~CSmSessionAssuranceCache()-[0x7FF6DEE06610] - D:\CA\siteminder\bin\smpolicysrv.exe
CSmPolicyExtensionCache::~CSmPolicyExtensionCache()-[0x7FF6DEE47C00] - D:\CA\siteminder\bin\smpolicysrv.exe
CServer::ProcessRequest()-[0x7FF9E504A580] - D:\CA\siteminder\bin\SMUTILITIES.dll
ThreadPool::Run()-[0x7FF9E5033A20] - D:\CA\siteminder\bin\SMUTILITIES.dll
ThreadPool::Run()-[0x7FF9E5033A20] - D:\CA\siteminder\bin\SMUTILITIES.dll
ThreadPoolBase::ThreadProc()-[0x7FF9E5092EF0] - D:\CA\siteminder\bin\SMUTILITIES.dll
o_exp()-[0x7FF9F2D22630] - C:\Windows\System32\ucrtbase.dll
BaseThreadInitThunk()-[0x7FF9F5B97960] - C:\Windows\System32\KERNEL32.DLL
RtlUserThreadStart()-[0x7FF9F5FAA2D0] - C:\Windows\SYSTEM32\ntdll.dll

Environment

Release : 12.8.06

Component : SiteMinder Federation(Federation Manager)

Symantec Adapter Version 9.1.1

Resolution

Broadcom engineering provided a patch, replacement of smauthhtml.dll on policy server side bin folder.  See attached smauthhtml_12806_win64bit.zip.

Alternatively, customer can upgrade to 12.8.07 release, which includes the fix.

Additional Information

DE541337

Advanced Auth Case #33174254

SiteMinder case #33180048