SiteMinder Agent Configuration on IIS 7.5 : kernel mode authentication

Products

• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• CA Single Sign-On

Issue/Introduction

Description:

When configuring the siteminder webagent 12SP3 with IIS 7.5 / Windows 2008 R2 with application pool in classic mode authentication / authorization request are going through the webagent and the policy server and user are correctly authenticated and authorize but we are still getting a windows popup.

Solution:

IIS 7.0 Kernel-mode authentication is enabled by default in IIS 7.0. This improves the performance of Windows Authentication, and simplifies the deployment of Kerberos authentication protocol. However, it may cause some clients that send the windows credentials on the initial request to fail due to a design limitation in kernel-mode authentication. Normal browser clients are not affected because they always send the initial request anonymously.

To over come this the solution is to disable the kernel-mode authentication during classic configuration of IIS.
Disable kernel-mode authentication by setting the userKernelMode to "false" in the system.webServer/security/authentication/windowsAuthentication section.
We can also do it by AppCmd as follows:
> %windir%system32inetsrvappcmd set config /section:windowsAuthentication /useKernelMode:false

Environment

Release:
Component: SMPLC