Certificate unknown response after upgrading to 11.2

Products

• CA API Gateway

Issue/Introduction

Mutual TLS connections are failing with a certificate_unknown message after upgrading from 11.1.3 to 11.2.

Environment

API Gateway 11.2

Cause

Enabling post-quantum encryption (BCJSEE) in system.properties caused a certificate check for client authentication extended usage. The connection fails if the certificate lacks the client authentication EKU.

Resolution

This issue appears only when Post-Quantum Encryption (BCJSSE) is enabled after upgrading to 11.2. The Post-Quantum Crypto Key Exchange can be enabled by adding the following line to the `system.properties` file:

• com.l7tech.security.tls13Provider=BCJSSE

To resolve the certificate unknown error, add the following lines *in addition* to the one above:

• org.bouncycastle.jsse.keyManager.checkEKU=false
• org.bouncycastle.jsse.trustManager.checkEKU=false