"Failed to decrypt persistent key" errors after upgrading the policy Server from v6.0 SP5 to R12 SP1/SP2/SP3.

Products

• CA Single Sign On Secure Proxy Server (SiteMinder)
• CA Single Sign On SOA Security Manager (SiteMinder)
• CA Single Sign-On

Issue/Introduction

Description:

After upgrading the SiteMinder Policy Server from v6.0 SP5 to R12 SP1/SP2/SP3; Policy server starts giving - "Failed to decrypt persistent key" errors in Policy Server (SMPS) logs.

++++++++++Error from SMPS Log ++++++++++++++
[20369/10][Tue Nov 16 2010 14:50:33][SmObjKeyManagement.cpp:457][ERROR]
Failed to decrypt persistent key
+++++++++++++++++++++++++++++++++++++++++

Cause of this Error:

Encryption key on file (EncryptionKey.txt) in a PS installation is used to generate the Policy Store key. With this key the Persistent Key is encrypted. Later, when PS needs the persistent key (to encrypt other data, like password blob or to do host registration) it uses the Policy Store Key to decrypt the **encrypted** Persistent Key which is the DB.

In 6.x policy servers, when PS couldn't decrypt this persistent key using the Policy Store Key, it would continue and use an empty PersistentKey to encrypt the data.

In r12.x Policy Servers, a test was added in the code to check if we succeeded to decrypt the PersistentKey, and return an error if we could not.

Such a situation can occur if, for example, the EncryptionKey.txt file was changed or copied from another machine. There can be other scenarios that can cause this to fail.

As a result, when a customer has this situation and doesn't know about it (that they use empty PersistentKey) and then upgrade to r12, they problem arises since in r12 the check is made to verify that the PersistentKey can be decrypted.

Solution:

IMPORTANT: This article contains information about modifying the registry. Before you modify the registry, make sure to create back up of the registry and ensure that you understand how to restore the registry if a problem may occur. For more information about how to back up, restore, and edit the registry, please review the relevant Microsoft Knowledge Base articles on support.microsoft.com.

 

Environment

Release:
Component: SMPLC

Resolution

This issue has been fixed in Policy Server version R12 SP2 and above.

A new Registry key (AllowEmptyEncKey) has been added for r12.x policy servers to prevent this error and work in the same way as 6.x policy server's work.

If this registry key is set (AllowEmptyEncKey=1), then -

If policy server cannot decrypt the PersistentKey; it will not log the "Failed to Decrypt Persistent Key" error in SMPS logs and will continue to use empty Persistent Key to encrypt the data (i.e. R12.x PS will keep on working like the 6.x one)

Registry Key:

HKEY_LOCAL_MACHINE\SOFTWARE\Netegrity\SiteMinder\CurrentVersion\ObjectStore
DWORD key: AllowEmptyEncKey
Value: 1

Note: By Default this key is not set.