Unable to load LLAWP in IIS HttpPlugin.dll

Products

• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• CA Single Sign-On

Issue/Introduction

In our Windows environment, I migrated to Policy Server v 12.8 SP2.  I have a 12.52 sp1 cr9 web agent installed on the policy server.
I receive a 500 error every time I attempt to load our homepage. The following is in the web agent logs every time IIS attempts to load LLAWP:

[3324/3492][Thu Mar 07 2019 16:47:51][CSmPluginManager.cpp:187][ERROR][sm-AgentFramework-00120] ADMIN: SmDSOLoadLibrary failed to load plugin 'D:\Program Files\CA\webagent\win64\bin\HttpPlugin.dll'.
[3324/3492][Thu Mar 07 2019 16:47:51][CSmPluginManager.cpp:96][ERROR][sm-AgentFramework-00110] ADMIN: Unable to load plugin 'D:\Program Files\CA\webagent\win64\bin\HttpPlugin.dll'.
[3324/3492][Thu Mar 07 2019 16:47:51][CSmAdminManager.cpp:191][ERROR][sm-AgentFramework-00270] ADMIN: Unable to get plugin context from the Plugin Manager.
[3324/3492][Thu Mar 07 2019 16:47:51][CSmHighLevelAgent.cpp:122][ERROR][sm-AgentFramework-00400] HLA: Failed to initialize 'Administration Manager'.
[6300/6292][Thu Mar 07 2019 17:09:06][LLAWorkerProcess.cpp:1962][INFO][sm-AgentFramework-00690] LLAWP: Stopping.

Environment

Lower dev/test environments:
Policy server - 12.8 sp2 Windows
Web Agent - 12.52 SP1 CR9 Windows (installed on the same machine as the policy server)

Cause

Usually, the issue is not actually with the ...\webagent\win64\bin\HttpPlugin.dll library, but rather it is caused by dependencies of HttpPlugin.dll that are not loaded properly.

Using Procmon, we are able to see what libraries are being loaded when the Web Agent tries to initialize.  Below are some examples.

We can see that HTTPPlugin.dll is properly loaded from the ...webagent\win64\bin directory.

Now we see the agent trying to load smvariable.dll.  The proper load should come from the webagent\win64\bin folder as we are using the 64bit web agent.  From the image, you can see that the w3wp process is actually loading smvariable.dll from the Policy Server bin directory.  This is not correct.

Here is another load that is found and loaded in the Policy server bin directory.  This is also not correct as it should be loaded from the webagent\win64\bin directory.

If we check our PATH environment variable, we can see that webagent\win64\bin is listed AFTER the policy server paths.  This is causing our problems.

Path=D:\Program Files\CA\webagent\win32\bin;D:\Program Files\CA\webagent\win64\install_config_info\lib;D:\Program Files\CA\siteminder\bin;D:\Program Files\CA\siteminder\bin\thirdparty;D:\Program Files\CA\siteminder\lib;D:\CA_JRE\bin;D:\Program Files\CA\webagent\win64\bin;C:\Windows\system32

Resolution

To resolve this issue, edit the PATH environment variable and move your "...\webagent\win64\bin" path to the second path in the list.  The important part is to have the "...\webagent\win64\bin" path in front of the policy server bin path.

For our example, below is the before and after: 

BEFORE: 
Path=D:\Program Files\CA\webagent\win32\bin;D:\Program Files\CA\webagent\win64\install_config_info\lib;D:\Program Files\CA\siteminder\bin;D:\Program Files\CA\siteminder\bin\thirdparty;D:\Program Files\CA\siteminder\lib;D:\CA_JRE\bin;D:\Program Files\CA\webagent\win64\bin;C:\Windows\system32 

AFTER: 
Path=D:\Program Files\CA\webagent\win32\bin;D:\Program Files\CA\webagent\win64\bin;D:\Program Files\CA\webagent\win64\install_config_info\lib;D:\Program Files\CA\siteminder\bin;D:\Program Files\CA\siteminder\bin\thirdparty;D:\Program Files\CA\siteminder\lib;D:\CA_JRE\bin;C:\Windows\system32

**NOTE: If you start the web agent and you still see the same error, you may need to reboot the machine so that the new environment variables are picked up**

Additional Information

This KB only refers to one of the many possible reasons for seeing the "ADMIN: SmDSOLoadLibrary failed to load plugin 'D:\Program Files\CA\webagent\win64\bin\HttpPlugin.dll'."  error message.

For other use cases, please check out the following KBs:

52514 - Rouge DLL files in windows system folders will cause the webagent to fail.

https://ca-broadcom.wolkenservicedesk.com/external/article?articleId=52514

53697 - Unable to load HTTPPLUGIN.DLL due to incompatible DLL files.

https://ca-broadcom.wolkenservicedesk.com/external/article?articleId=53697