Java applications encounter SSLHandshakeException/CertificateException after upgrading to Netty 4.2.x

Products

• VMware Tanzu Spring Runtime

Issue/Introduction

After upgrading the Netty library from 4.1.x to 4.2.x (or upgrading a framework that includes Netty 4.2, such as certain Spring Boot versions), applications may fail to establish TLS connections to remote endpoint with the error:

javax.net.ssl.SSLHandshakeException: General SSLEngine problemCaused by: java.security.cert.CertificateException: No subject alternative DNS name matching <hostname> found.

Cause

In Netty version 4.2, the default endpointIdentificationAlgorithm in SslContext was changed from NONE to HTTPS.

Previously, in version 4.1, Netty did not perform hostname verification by default unless explicitly configured. In 4.2, hostname verification is enabled by default. If the server’s certificate does not have a Subject Alternative Name (SAN) that matches the hostname used by the client to connect, the handshake will fail.

Resolution

There are two primary ways to resolve this issue:

Option 1: Update the Server Certificate (Recommended) Update the server-side TLS certificate to include the correct hostname in the Subject Alternative Name (SAN) field. This is the most secure approach as it maintains the integrity of hostname verification.

Option 2: Use the JVM System Property (Global Workaround) If you cannot update the certificate and wish to restore the Netty 4.1 behavior, you can disable the default endpoint verification by adding the following system property to your application's startup command: 

-Dio.netty.handler.ssl.defaultEndpointVerificationAlgorithm=NONE