"XCOMU0780E ... TxpiInitSSL ... EVP_DecryptFinal_ex:bad decrypt"

Products

• XCOM Data Transport
• XCOM Data Transport - Linux PC
• XCOM Data Transport - Windows
• XCOM Data Transport - z/OS

Issue/Introduction

XCOM for Linux 11.6 SP01 SSL loopback test fails with these messages in xcom.log and trace file (XTRACE=10):
===
XCOMU0780E Txpi  308: TxpiInitSSL Failed msg = <error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt> value = 0:

...
#XCOMU0298E Unable to allocate remote transaction program: Txpi  227: Socket received 0 bytes: partner closed socket. Last error: 0
===

Environment

Release : 11.6

Resolution

Comparing the used $XCOM_HOME/config/configssl.cnf to a default version a difference was found in the [PRIVATEKEY] section
===
[PRIVATEKEY]
INITIATE_SIDE = $XCOM_HOME/ssl/private/casslkey.pem
RECEIVE_SIDE  = $XCOM_HOME/ssl/private/serverkey.pem
===
The INITIATE_SIDE should normally be $XCOM_HOME/ssl/private/clientkey.pem
After making that change to use clientkey.pem the loopback test was then successful.