SSLProtocolException: handshake alert: unrecognized_name

book

Article ID: 140549

calendar_today

Updated On:

Products

CA Single Sign On Secure Proxy Server (SiteMinder) AXIOMATICS POLICY SERVER CA Single Sign On Agents (SiteMinder) CA Single Sign On Federation (SiteMinder) CA Single Sign On SOA Security Manager (SiteMinder) SITEMINDER

Issue/Introduction

Customer is getting javax.net.ssl.SSLProtocolException: handshake alert:  unrecognized_name when accessing internal URL.

Environment

Release : 12.8

Component : SITEMINDER -POLICY SERVER

Resolution

Disable SNI:

 

1. Stop SPS/Access Gateway


2. Make a backup copy of CA\secure-proxy\proxy-engine\conf\SmSpsProxyEngine.properties


3. Edit CA\secure-proxy\proxy-engine\conf\SmSpsProxyEngine.properties and find:


NETE_SPS_PROXYENGINE_CMD="%NETE_SPS_JAVA_HOME%\bin\java.exe" -Xms512m -Xmx1024m -XX:MaxMetaspaceSize=256M -Dcatalina.base="%NETE_SPS_TOMCAT_HOME%" -Dcatalina.home="%NETE_SPS_TOMCAT_HOME%" -Djava.endorsed.dirs="%NETE_SPS_TOMCAT_HOME%\endorsed" -Djava.io.tmpdir="%NETE_SPS_TOMCAT_HOME%\temp" -DHTTPClient.log.mask=0 -DHTTPClient.Modules="HTTPClient.RetryModule|org.tigris.noodle.NoodleCookieModule|HTTPClient.DefaultModule" -DSM_AGENT_LOG_CONFIG="%STS_AGENT_LOG_CONFIG_FILE%" -Dfile.encoding=UTF8 -DIWACONFIGHOME="%IWACONFIGHOME%" -Dlogger.properties="%NETE_SPS_TOMCAT_HOME%\properties\logger.properties" -classpath "%NETE_SPS_TOMCAT_HOME%\bin\proxybootstrap.jar;%NETE_SPS_TOMCAT_HOME%\properties;%NETE_SPS_JAVA_HOME%\lib\tools.jar;%NETE_SPS_TOMCAT_HOME%\bin\bootstrap.jar;%NETE_SPS_ROOT%\resources;%NETE_SPS_ROOT%\agentframework\java\bc-fips-1.0.1.jar" com.netegrity.proxy.ProxyBootstrap -config "%NETE_SPS_ROOT%/proxy-engine/conf/server.conf"


4. Modify NETE_SPS_PROXYENGINE_CMD to add -Djsse.enableSNIExtension=false like the below.

 

NETE_SPS_PROXYENGINE_CMD="%NETE_SPS_JAVA_HOME%\bin\java.exe" -Xms512m -Xmx1024m -XX:MaxMetaspaceSize=256M -Dcatalina.base="%NETE_SPS_TOMCAT_HOME%" -Dcatalina.home="%NETE_SPS_TOMCAT_HOME%" -Djava.endorsed.dirs="%NETE_SPS_TOMCAT_HOME%\endorsed" -Djava.io.tmpdir="%NETE_SPS_TOMCAT_HOME%\temp" -DHTTPClient.log.mask=0 -DHTTPClient.Modules="HTTPClient.RetryModule|org.tigris.noodle.NoodleCookieModule|HTTPClient.DefaultModule" -DSM_AGENT_LOG_CONFIG="%STS_AGENT_LOG_CONFIG_FILE%" -Dfile.encoding=UTF8 -DIWACONFIGHOME="%IWACONFIGHOME%" -Dlogger.properties="%NETE_SPS_TOMCAT_HOME%\properties\logger.properties" -Djsse.enableSNIExtension=false -classpath "%NETE_SPS_TOMCAT_HOME%\bin\proxybootstrap.jar;%NETE_SPS_TOMCAT_HOME%\properties;%NETE_SPS_JAVA_HOME%\lib\tools.jar;%NETE_SPS_TOMCAT_HOME%\bin\bootstrap.jar;%NETE_SPS_ROOT%\resources;%NETE_SPS_ROOT%\agentframework\java\bc-fips-1.0.1.jar" com.netegrity.proxy.ProxyBootstrap -config "%NETE_SPS_ROOT%/proxy-engine/conf/server.conf"


5. Start SPS/Access Gateway

Powered by Wolken Software