New IBM JVM Security Patch (SR6 FP25 or newer) Issue with Broadcom SAAS Certificate
search cancel

New IBM JVM Security Patch (SR6 FP25 or newer) Issue with Broadcom SAAS Certificate

book

Article ID: 217567

calendar_today

Updated On:

Products

CA Application Performance Management SaaS

Issue/Introduction

With Java version "1.8.0_281"
Java(TM) SE Runtime Environment (build 8.0.6.25 - pxa6480sr6fp25-20210115_01(SR6 FP25))

Receiving this error when trying to connect an agent to APM SAAS:

[IntroscopeAgent.ConnectionThread] Failed to connect to the Introscope Enterprise Manager at apmgw.dxi-na1.saas.broadcom.com:443,com.wily.isengard.client.transport.websocket.SSLWebSocketFactory (io.netty.handler.codec.DecoderException:javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure)

 

Changing this file:

......./jre/lib/security/java.security

From this:

# # List of providers and their preference orders (see above): # security.provider.1=com.ibm.jsse2.IBMJSSEProvider2 security.provider.2=com.ibm.crypto.provider.IBMJCE security.provider.3=com.ibm.crypto.plus.provider.IBMJCEPlus security.provider.4=com.ibm.security.jgss.IBMJGSSProvider security.provider.5=com.ibm.security.cert.IBMCertPath security.provider.6=com.ibm.security.sasl.IBMSASL security.provider.7=com.ibm.xml.crypto.IBMXMLCryptoProvider security.provider.8=com.ibm.xml.enc.IBMXMLEncProvider security.provider.9=com.ibm.security.jgss.mech.spnego.IBMSPNEGO security.provider.10=sun.security.provider.Sun

To this:

# # List of providers and their preference orders (see above): # security.provider.1=com.ibm.jsse2.IBMJSSEProvider2 security.provider.2=com.ibm.crypto.provider.IBMJCE #security.provider.3=com.ibm.crypto.plus.provider.IBMJCEPlus security.provider.3=com.ibm.security.jgss.IBMJGSSProvider security.provider.4=com.ibm.security.cert.IBMCertPath security.provider.5=com.ibm.security.sasl.IBMSASL security.provider.6=com.ibm.xml.crypto.IBMXMLCryptoProvider security.provider.7=com.ibm.xml.enc.IBMXMLEncProvider security.provider.8=com.ibm.security.jgss.mech.spnego.IBMSPNEGO security.provider.9=sun.security.provider.Sun

Fixes the issue, it seems like a new security library that IBM JVM is using as of that patch (security.provider.3=com.ibm.crypto.plus.provider.IBMJCEPlus) is interacting poorly with something on the APM SAAS side. Is there something about your SSL implementation or cert that can change to support this change by IBM?

 

Environment

Release : 3.4

Component :

Cause

Cipher 

Resolution

com.ibm.crypto.plus.provider.IBMJCEPlus/TLS1.3 issue with the IBM JVM Security Patch SR6 FP25:

 

-------------------------------------------------------------------------------------------

 

SSL related troubleshooting steps in case the handshake fails due to cipher mismatch:

There are many reasons why APM Agents may fail to connect to SaaS service. Most of the time the problem is misspelled hostname or port, firewall connectivity issues, or incorrect or out-of-date credentials. This support article covers specific troubleshooting steps where all former issues are ruled out and the problem with agent connection SSL persists.

 

The indication that there is an issue with SSL handshake is following log in agent log:

 

 [ERROR] [IntroscopeAgent.IsengardMessaging] WebSocket Client caught io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Received fatal alert: handshake_failure

 

, or when using https protocol instead of websocket: 

 

[WARN] [IntroscopeAgent.CommonsHttpTunnelingClient] connectBinary failed to perform SSL handshake: Received fatal alert: handshake_failure

 

A most common reason for SSL handshake failure is a mismatch of configured or supported cryptographic ciphers in the JVM hosting agent and the allowed ciphers in our APM SaaS service. 

 

As new vulnerabilities are discovered, our SaaS service endpoint is periodically updated, and the ciphers that were deemed secure before may no longer allowed. In that case older cipher settings in agents running on older JVMs may stop working. In some cases is possible to find and configure new ciphers on agent side. 

 

This involves:

  1. Identifying the set of ciphers that are supported by server.
  2. Identifying the set of ciphers that are supported by JVM that runs agent.
  3. Finding a set of ciphers that is common between server and agent.
  4. Configuring ciphers in agent profile using agentManager.cipherSuites.1 property.

 

How to identify set of ciphers that APM SaaS server supports?

 

There are several opensource SSL analytics tools that scan a service endpoint and list supported ciphers. One of the easiest to use is an SSL Labs online tool:

 

https://www.ssllabs.com/ssltest/index.html

 

Just enter the host name and run report. Once report is ready scroll down to the Cipher Suites section and note down the supported ciphers.

 

NOTE: If using some other tool than above, note that OpenSSL based tools often use different naming scheme than Java and java tools use (IANA naming convention). Following site provides mapping between open SSL and IANA:

 

https://testssl.sh/openssl-iana.mapping.html

 

How to identify set of ciphers that JVM running agent supports?

 

When agent connects to SaaS APM Service for the first time it lists all ciphers supported by JVM into the agent log:

 

[INFO] [IntroscopeAgent.Agent] SSL socket factory supported ciphers are: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, ...  ]

 

It's usually quite a long list.

How to find a common set?

Finding a supported common set is usually easy by just doing a string comparison of the two sets and keeping strings that match, but there are two special cases.

 

First if OpenSSL based tool is used to identify server ciphers they need to be converted to IANA format. So for example, if tool gives supported cipher as ECDHE-ECDSA-AES128-SHA256 it should be entered as TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA25. See mapping site above in step 1.

 

Second IBM Java JVM uses a different prefix than Oracle, OpenDJK and the rest of the free world. You will notice that all supported ciphers in IBM JVM start with SSL_* prefix, while the server-supported ciphers have usually TLS_*. The good news is that just changing the prefix from TLS_* to SSL*. works. For example, if server supports TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA25 cipher and JVM supports SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA25 cipher, then use SSL_ECDHE_ECDSA_WITH_AES_128_CBC_SHA25 in the next step as part of common set.

 

How to configure cipher set to be used by the agent?

 

You need to configure the set that you determined in last step to agentManager.cipherSuites.<n> property, where <n> is the order of alternative service endpoints. Most of the time there is just one endpoint so <n> is 1.

 

For example find and set in IntroscopeAgent.profile:

 

agentManager.cipherSuites.1=TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

The property can also be configured from JVM command line if needed.

 

 

What to do when there are no common ciphers between JVM and server?

There is an alternative to install our APM Cloud Proxy and configure agents to use plain unencrypted TCP connections through proxy.

-----------------------------------------------------------------------------------------------------

Powered by Wolken Software