TLS Listener fails with error "ORA-28864: SSL connection closed gracefully TNS-12560: TNS:protocol adapter error TNS-00542: SSL Handshake failed 64-bit Windows Error: 28864: Unknown error" after enabling TCPS

Products

• Data Loss Prevention Oracle Standard Edition 2

Issue/Introduction

When enabling TCPS on the listener you notice the following error when starting the listener:

LSNRCTL for 64-bit Windows: Version 19.0.0.0.0 - Production on 25-AUG-2023 12:44:15

Copyright (c) 1991, 2020, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCPS)(HOST=MyOracleServer)(PORT=2484)))
ORA-28864: SSL connection closed gracefully
 TNS-12560: TNS:protocol adapter error
  TNS-00542: SSL Handshake failed
   64-bit Windows Error: 28864: Unknown error
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=protect)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for 64-bit Windows: Version 19.0.0.0.0 - Production
Start Date                25-AUG-2023 12:44:02
Uptime                    0 days 0 hr. 0 min. 13 sec
Trace Level               support
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   C:\Oracle\Product\19.3.0.0\db_1\network\admin\listener.ora
Listener Log File         C:\oracle\diag\tnslsnr\Oracle16\listener\alert\log.xml
Listener Trace File       C:\oracle\diag\tnslsnr\Oracle16\listener\trace\ora_2396_6372.trc
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=MyOracleServer)(PORT=2484)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\protectipc)))
Services Summary...
Service "CLRExtProc" has 1 instance(s).
  Instance "CLRExtProc", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

Environment

Release : 19

Cause

This can be caused by lack of permissions on the wallet files.  So when you create a wallet and certs using orapki utility, by default it looks like it disables inheritance after creating the files under the wallet directory.  The user which is used to run the listener service in services.msc needs to have full permissions over this directory.

Resolution

Fix permissions or enable inheritance on all files under the wallet folder.