How to disable KeyManager and TrustManager in Container Security Provider Framework in Java buildpack
Article ID: 297451
Updated On: 02-07-2025
Products
Issue/Introduction
This article explains how to disable the KeyManager and/or TrustManager in the Container Security Provider Framework in Java buildpack.
The Container Security Provider Framework, introduced in Java buildpack 3.17, adds a Security Provider to the Java Virtual Machine (JVM) that automatically includes BOSH trusted certificates, Diego identity certificates, and private keys.
In some cases, you may want to disable either the KeyManager, the TrustManager, or both.
Oftentimes, the Container Security Provider will do the right thing and work properly. There are a few occasions where it can cause problems. The most common is with KeyManager only.
For example, if you have a Java application running in Tanzu Application Service for VMs (TAS for VMs) Foundation A that is accessing another app running in TAS for VMs Foundation B, you may encounter issues if TAS for VMs Foundation B is configured to support mutual Transport Layer Security (TLS) and validate client certificates. Foundation B does not need to be configured to require mutual TLS certificates, this issue will happen even if Foundation B is configured with mutual TLS certificates as optional.
The Container Security Provider will automatically add the Diego issued application instance ID cert to outbound requests. Regardless of the way Gorouter is configured, required or optional, it will always validate a mutual TLS cert if the client includes one. The Container Security Provider always includes a cert and the Gorouter will always validate it. By default, Foundation B is not configured to trust Diego Instance ID certs issued by Foundation A and you will get the error, "bad_certificate".
An alternative to disabling KeyManager is the set the option "Gorouter does not request client certificates" in TAS > Networking configuration. This option will make it so router does not request and validate the Diego Instance ID cert for application on Foundation A.
Issues with TrustManager are much less common and you generally want to keep this enabled, even when disabling KeyManager. TrustManager is responsible for automatically making sure that the JVM trusts any BOSH deployed trusted TLS certificate authorities.
In most cases, you can trust these. However, if trusting them will create problems or if you need to be more restrictive about the certificate authorities that you trust, you can disable TrustManager.
Resolution
In order to disable KeyManager, use the application environment variable JBP_CONFIG_CONTAINER_SECURITY_PROVIDER. Set the key_manager_enabled key to false. For example:
cf set-env <JAVA-APP-NAME> JBP_CONFIG_CONTAINER_SECURITY_PROVIDER '{key_manager_enabled: false}'
In order to disable
TrustManager, use the same application environment variable JBP_CONFIG_CONTAINER_SECURITY_PROVIDER and set the trust_manager_enabled key to false. For example:
cf set-env <JAVA-APP-NAME> JBP_CONFIG_CONTAINER_SECURITY_PROVIDER '{trust_manager_enabled: false}'
To disableKeyManagerandTrustManagerat the same time, refer to the example below:
cf set-env <JAVA-APP-NAME> JBP_CONFIG_CONTAINER_SECURITY_PROVIDER '{key_manager_enabled: false, trust_manager_enabled: false}'
Restage the application to ensure the change to environment variable that the buildpack consumes takes effect.
cf restage <JAVA-APP-NAME>
The staging process has access to environment variables, so the environment can affect the contents of the droplet. Restaging the application compiles a new droplet from your application without updating your app source.
For more information, refer to Container Security Provider.
Known Issue
A very rare bug has been identified within the Container Security Provider Framework that may contribute to communication errors for an application when reaching out to other endpoints via mutual TLS. An application's Diego Instance Identity certificate and key pair are reloaded every 24 hours and the Framework watches these files to update
the Security Provider accordingly. A low level concurrency bug makes it possible for a mismatching certificate and key pair to be loaded in the Security Provider and when this occurs TLS negotiations may fail until the application is restarted or the Instance Identity certificate and key are rotated again.The error state may show itself as "bad_certificate" or "handshake_failure".
Initial log for when the Instance Identity certificate and key are rotated:
o.c.s.FileWatchingX509ExtendedKeyManager.run - Updated KeyManager for /etc/cf-instance-credentials/instance.key and /etc/cf-instance-credentials/instance.crt
TLS failure Immediately following the rotation of the Instance Identity certificate and key when the error state is present:
I/O error on GET request for \""https://credhub.service.cf.internal:8844/api/v1/data\"": Received fatal alert: handshake_failure; nested exception is javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure]
I/O error on GET request for ""https://credhub.service.cf.internal:8844/api/v1/data"": readHandshakeRecord; nested exception is javax.net.ssl.SSLException: readHandshakeRecord] with root cause"
This issue has been fixed in Java buildpack v4.57. Until upgrade your apps with the fixed Java buildpack, the workaround is to restart the application instance facing this error.
Additional Information
Feedback
