SmCertDBMigrate Utility Execution Failed
search cancel

SmCertDBMigrate Utility Execution Failed

book

Article ID: 443129

calendar_today

Updated On:

Products

SITEMINDER

Issue/Introduction

During an upgrade of the Siteminder Policy Server to r12.9 the SmCertDBMigrate utility fails with the following error:

SMCertDBMigrate utility, which exports all certificates from the Netscape Certificate Database File to trusted.pem, failed with the following error:

COMMAND: "<Install_Dir>\CA\siteiminder\bin\SmCertDBMigrate.bat

STDERR: certutil: function failed: SEC_ERROR_BAD_DATABASE: security library: bad database.

STDOUT: <Install_Dir>\CA\siteminder\Files\CA\siteminder\certdb\

"Certificate Database File path is incorrect."

"Fix the reported error and run SmCertDBMigrate utility manually to migrate certificates to trusted.pem

RETURN: 1

 

Environment

PRODUCT: Symantec Siteminder

COMPONENT: Policy Server

VERSION: 12.9

OPERATING SYSTEM: Any

Cause

Starting in Siteminder r12.9 the Policy Server uses OpenLDAP  to communicate with an LDAP session store. For the rest of the LDAP stores such as policy store, key store, and user store, Policy Server continues to use Mozilla LDAP for the communication. Policy Server now uses the trusted.pem file at <siteminder-installation-path>/config for its SSL communication with an LDAP session store. The trusted.pem file contains all the root certificates in a Base-64 encoded format.

  • In a new installation of Release 12.9, OpenLDAP is used to communicate with an LDAP session store and Mozilla LDAP is used to communicate with the rest of the LDAP stores. To add any new certificate in the 12.9 environment, add the certificate to a certificate database and also to the trusted.pem file in a Base-64 encoded format. Ensure that all the certificates of all the LDAP stores are available in both the locations.
  • In an in-place upgrade from a previous release to Release 12.9, OpenLDAP replaces Mozilla LDAP to communicate with an LDAP session store. If Policy Server is already mapped with a certificate database, the upgrade process uses the newly introduced 
    SmCertDBMigrate
     utility in the background to seamlessly copy the existing certificates that are present in the certificate database into the trusted.pem file, in a Base-64 encoded format. To add a new certificate or update an existing certificate in the 12.9 environment, make the change in the certificate database and the trusted.pem file too. Ensure that all the certificates of all the LDAP stores are available in both the locations.
  • In a rolling upgrade, if you choose to copy an existing certificate database onto the latest Policy Server, you must run the 
    SmCertDBMigrate
     utility to seamlessly copy the existing certificates that are present in the certificate database into the trusted.pem file, in a Base-64 encoded format. This step ensures a seamless communication between Policy Server and an LDAP session store in the latest environment. To add a new certificate or update an existing certificate in the 12.9 environment, make the change in the certificate database and the trusted.pem file too. Ensure that all the certificates of all the LDAP stores are available in both the locations.
A review of the 'SmCertDBMigrate' will show the following:
 
LINUX
 
<Install_Dir>/CA/siteminder/bin/SmCertDBMigrate.sh
 
=========================================

certDBPath=$(smregistryupdate fetch CertDbPath | gawk -F"?" '{print $2}')

if [ "$certDBPath" = "" ];then
   echo "Certificate Database File path is empty."
   exit 0
fi

certDBPath="${certDBPath%/*}/"
outputFile="$NETE_PS_ROOT/config/trusted.pem"

certutil -L -d $certDBPath

if [ $? -ne 0 ]; then
    echo "Certificate Database File path is incorrect."
    echo "Fix the reported error and run SmCertDBMigrate utility manually to migrate certificates to trusted.pem."
    exit 1
fi

=========================================
 
WINDOWS
 
<Install_Dir>\CA\siteminder\bin\SmCertDBMigrate.bat
 
=========================================

set certDB=%certDB:"=%
echo %certDB%

if "%certDB%" == "" (
echo "Certificate Database File path is empty."
exit /b 0
)

certutil -L -d %certDB%

if %ERRORLEVEL% NEQ 0 (
echo "Certificate Database File path is incorrect."
echo "Fix the reported error and run SmCertDBMigrate utility manually to migrate certificates to trusted.pem."
exit /b 1
)

=========================================
 
Neither script is actually setting the environment variable [Windows: CertDB | Linux: CertDBPath].  The CertDB/CertDBPath environment variable is not set as part of the default Siteminder Environment variables.  
 
 
 
 

Resolution

There are several possible options to resolve this issue.

OPTION #1: Set the 'CertDB/CertDBPath' Environment Variable persistently for all sessions

This is the recommended method.

LINUX

1) Edit the Siteminder Policy Server Environment script

<Install_Dir>/CA/siteminder/ca_ps_env.ksh

2) Add the 'CertDBPath' environment variable

EXAMPLE: CertDBPath="$NETE_PS_ROOT/opt/CA/siteminder/CertDB"; export CertDBPath

################################################################################

NETE_PS_ROOT="/opt/CA/siteminder"; export NETE_PS_ROOT

CertDBPath="$NETE_PS_ROOT/<Path_to_CertDB_Directory>"; export CertDBPath
NETE_TOOL="$NETE_PS_ROOT/BitnessDetector"; export NETE_TOOL

...

3) Save the changes

4) Run the Policy Server Environment Variable Script

. ./ca_ps_env.ksh

 

WINDOWS

1) Open System Properties

2) Select the ADVANCED tab

3) 'Click' the "Environment Variables" button

4) Under "System Variables" 'click' the NEW... button.

5) Populate the following values:

Variable Name = CertDB

Variable Value = <Path_to_CertDB_Directory>

6) 'Click' OK to close and exit.

 

OPTION #2: Set the 'CertDB/CertDBPath' Environment Variable for the active session

LINUX

Run the following command:

export CertDBPath=<Path_to_CertDB_Directory>

WINDOWS

1) Run 'cmd.exe' with elevated privileges (Run As Administrator)

2) Run the following command:

set CertDB=<Path_to_CertDB_Directory>

NOTE: This will only apply to the active session.  This will only apply to manually running the 'SmCertDBMigrate' utility manually.

 

OPTION #3: Hardcode the path to the CertDB file in the 'SmCertDBMigrate' utility

LINUX

1) Edit the 'SmCertDBMigrate.sh' script.

<Install_Dir>/CA/siteminder/bin/SmCertDBMigrate.sh

2) Hard code the path to the CertDB directory

#certutil -L -d $certDBPath
certutil -L -d <Path_to_CertDB_Dir>

 

WINDOWS

1) Edit the 'SmCertDBMigrate.bat' script.

<Install_Dir>\CA\siteminder\bin\SmCertDBMigrate.sh

2) Hard code the path to the CertDB directory

#certutil -L -d %certDB%
certutil -L -d <Path_to_CertDB_Dir>

NOTE: There is no trailing "/" at the end of the CertDB Path Value.  The file (.i.e. cert9.db) is not defined in the environment variable

INCORRECT: <Install_Dir>/CA/siteminder/certdb/

INCORRECT: <Install_Dir>/CA/siteminder/certdb/cert9.db

CORRECT: <Install_Dir>/CA/siteminder/certdb

Additional Information

Change to the Communication between Policy Server and an LDAP Session Store