Customer is getting javax.net.ssl.SSLProtocolException: handshake alert: unrecognized_name when accessing internal URL.
Release : 12.8
Component : SITEMINDER -POLICY SERVER
Disable SNI:
1. Stop SPS/Access Gateway
2. Make a backup copy of CA\secure-proxy\proxy-engine\conf\SmSpsProxyEngine.properties
3. Edit CA\secure-proxy\proxy-engine\conf\SmSpsProxyEngine.properties and find:
NETE_SPS_PROXYENGINE_CMD="%NETE_SPS_JAVA_HOME%\bin\java.exe" -Xms512m -Xmx1024m -XX:MaxMetaspaceSize=256M -Dcatalina.base="%NETE_SPS_TOMCAT_HOME%" -Dcatalina.home="%NETE_SPS_TOMCAT_HOME%" -Djava.endorsed.dirs="%NETE_SPS_TOMCAT_HOME%\endorsed" -Djava.io.tmpdir="%NETE_SPS_TOMCAT_HOME%\temp" -DHTTPClient.log.mask=0 -DHTTPClient.Modules="HTTPClient.RetryModule|org.tigris.noodle.NoodleCookieModule|HTTPClient.DefaultModule" -DSM_AGENT_LOG_CONFIG="%STS_AGENT_LOG_CONFIG_FILE%" -Dfile.encoding=UTF8 -DIWACONFIGHOME="%IWACONFIGHOME%" -Dlogger.properties="%NETE_SPS_TOMCAT_HOME%\properties\logger.properties" -classpath "%NETE_SPS_TOMCAT_HOME%\bin\proxybootstrap.jar;%NETE_SPS_TOMCAT_HOME%\properties;%NETE_SPS_JAVA_HOME%\lib\tools.jar;%NETE_SPS_TOMCAT_HOME%\bin\bootstrap.jar;%NETE_SPS_ROOT%\resources;%NETE_SPS_ROOT%\agentframework\java\bc-fips-1.0.1.jar" com.netegrity.proxy.ProxyBootstrap -config "%NETE_SPS_ROOT%/proxy-engine/conf/server.conf"
4. Modify NETE_SPS_PROXYENGINE_CMD to add -Djsse.enableSNIExtension=false like the below.
NETE_SPS_PROXYENGINE_CMD="%NETE_SPS_JAVA_HOME%\bin\java.exe" -Xms512m -Xmx1024m -XX:MaxMetaspaceSize=256M -Dcatalina.base="%NETE_SPS_TOMCAT_HOME%" -Dcatalina.home="%NETE_SPS_TOMCAT_HOME%" -Djava.endorsed.dirs="%NETE_SPS_TOMCAT_HOME%\endorsed" -Djava.io.tmpdir="%NETE_SPS_TOMCAT_HOME%\temp" -DHTTPClient.log.mask=0 -DHTTPClient.Modules="HTTPClient.RetryModule|org.tigris.noodle.NoodleCookieModule|HTTPClient.DefaultModule" -DSM_AGENT_LOG_CONFIG="%STS_AGENT_LOG_CONFIG_FILE%" -Dfile.encoding=UTF8 -DIWACONFIGHOME="%IWACONFIGHOME%" -Dlogger.properties="%NETE_SPS_TOMCAT_HOME%\properties\logger.properties" -Djsse.enableSNIExtension=false -classpath "%NETE_SPS_TOMCAT_HOME%\bin\proxybootstrap.jar;%NETE_SPS_TOMCAT_HOME%\properties;%NETE_SPS_JAVA_HOME%\lib\tools.jar;%NETE_SPS_TOMCAT_HOME%\bin\bootstrap.jar;%NETE_SPS_ROOT%\resources;%NETE_SPS_ROOT%\agentframework\java\bc-fips-1.0.1.jar" com.netegrity.proxy.ProxyBootstrap -config "%NETE_SPS_ROOT%/proxy-engine/conf/server.conf"
5. Start SPS/Access Gateway