Unable to register Layer7 API Gateway with Policy Server

Products

• SITEMINDER
• CA Single Sign On Agents (SiteMinder)
• CA Single Sign On Federation (SiteMinder)
• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign-On
• CA Single Sign On SOA Security Manager (SiteMinder)

Issue/Introduction

Layer 7 admin is unable to register Layer7 API Gateway policy manager with SiteMinder Policy Server.

Layer7 API Gateway policy manager provides UI interface, where trusted host can be registered with SiteMinder Policy Server.

Required input fields are:

Address:
Hostname:
Host Configuration:
FIPS Mode:
User Name:
Password:

All data values are provided, when hit "register" button, gets an error:

"Registration failed: Unable to invoke the smreghost program".

Siteminder smps.log has this error:

[SmObjKeyManagement.cpp:459][ERROR][sm-Server-03080] Failed to decrypt persistent key

 

Environment

SiteMinder Policy Server 12.8SP6
Layer7 API Gateway 10.1.00-11620

 

Cause

Either persistent key is out of sync among different policy servers or persistent key value is somehow empty.

This can be verified by running the command:

  smkeyexport -d<adminname> -w<password> -o<keyfilename> -c

on each Policy Server, then compare output file content.

 

Resolution

 

Setting/adding AllowEmptyEncKey in sm.registry file or Windows registry directly resolved the issue.

REGISTRY KEY:
HKEY_LOCAL_MACHINE\SOFTWARE\Netegrity\SiteMinder\CurrentVersion\ObjectStore
DWORD key: AllowEmptyEncKey
Value: 1

AllowEmptyEncKey instructs Policy Server to use an empty persistent key to encrypt policy store data if Policy Server fails to decrypt the persistent key from the Key Store.

0 - Disable
1 - Enable

In addition, when a single Policy Server generates encryption keys in an environment with multiple Policy Servers that connect to disparate Policy Stores, but share a central Key Store, an additional registry setting is required. This registry setting configures each Policy Server to poll the common Key Store and retrieve new encryption keys at a regular interval.

Change the following registry value:

"EnableKeyUpdate"=0

to

"EnableKeyUpdate"=1

Restart the Policy Server.

 

Additional Information

 

1. Manage the Session Ticket Key
   https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/siteminder/12-8/administrating/manage-encryption-keys/manage-the-session-ticket-key.html