Cannot route to an protected service with SSL/TLS: Certificates does not conform to algorithm constraints. Certificates does not conform to algorithm constraints
search cancel

Cannot route to an protected service with SSL/TLS: Certificates does not conform to algorithm constraints. Certificates does not conform to algorithm constraints

book

Article ID: 42925

calendar_today

Updated On:

Products

STARTER PACK-7 CA Rapid App Security CA API Gateway

Issue/Introduction

Solution

Background

The CA API Gateway ensures the security of message traffic by requiring that protected services using SSL or TLS?are using suitably strong hashing algorithms?and use adequately long private key sizes. Attempts to consume a protected service that uses a server certificate that is issued by a certificate authority?using weak algorithms or small?keys in the issuing certificate?may?be rejected under a specific set of circumstances.?Message rejection will occur when:

  1. The Sun Java Secure Socket Extension cryptographic suite is in use.
  2. An issuing certificate does not adhere to a specific set of algorithm constraints

This article will prescribe any necessary changes to address this behavior.

Presentation

The following error message may be present in the Gateway audits or logs. The presence of this message indicates that the issuer of a server certificate is not adhering to a specific

Problem routing to https://service.domain.com/path/to/service.?Error msg: Unable to obtain HTTP response from https://service.domain.com/path/to/service: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints. Caused by: Certificates does not conform to algorithm constraints

This message indicates that the issuer of a server certificate is using disallowed settings such as hash algorithm or key length. The default setting of the Gateway disallows issuer certificates that use MD2 as a hashing algorithm or that have a key length of less than 1024 bits. For example--Server certificates that are stored in the?Manage Certificates?trust store of the Gateway may have the following displayed values:

Signature algorithm: MD2withRSA
RSA strength: 512 bits

Troubleshooting

If the issue is presenting as documented above then?it will be necessary to gather some diagnostic data before determining how to resolve this issue. Two system configuration files may contain directives that directly impact the observed behavior. If both of the following files contain the specified directive and value then the Gateway will need to be reconfigured to accommodate. Make a note of these settings as they will be used in later steps.

FileDirectiveValue
/opt/SecureSpan/Gateway/node/default/etc/conf/system.propertiescom.l7tech.security.tlsProviderSunJSSE
/opt/SecureSpan/JDK/jre/lib/security/java.securityjdk.certpath.disabledAlgorithmsMD2, RSA keySize < 1024

There are three resolution paths that can be leveraged to address this issue:

  1. Add an additional level of trust to the?Manage Certificates?trust store
  2. Modify the security settings of the Gateway JDK to allow a disabled algorithm
  3. Disabling the use of SunJSSE as a cryptographic suite

Resolution

Any one of the three changes recommended in the previous section can resolve the issue. Adding an additional level of trust above the impacted server certificate will allow the Gateway to establish trust regardless of any disabled algorithms or key lengths. Avoiding the use of SunJSSE will result in the algorithm restriction being unenforced. Modifying the security settings of the JDK will make the rules more permissive for issuing certificates.

Adding trust levels

This solution is predicated on the fact that an issuer certificate has a parent issuer. Some commercial certificate authorities or government agencies leverage?multi-tiered certificate authority infrastructures. If an issuing certificate has a parent issuer that does not violate the constraints specified previously then the higher issuing certificate can be added. Perform the following procedure to add a higher issuing certificate and trust it as a trust anchor

  1. Log in the CA API Policy Manager as an administrative user
  2. Open the?Manage Certificates?task from the?Task/Manage?menu
  3. Select the existing issuer certificate and click?Properties
  4. Navigate to the?Validation?tab
  5. Disable?Certificate is a Trust Anchor
  6. Save changes and exit the dialog
  7. Select?Add?to add a new certificate
  8. Navigate through the wizard to import the new issuer certificate
  9. Ensure the Signature Algorithm is not?MD2?or any other disallowed algorithm specified in java.security
  10. Ensure the RSA Strength value is greater than or equal to?1024?or?exceeds the minimum key length specified in?java.security
  11. Enable?Signing Client Certificates for Outbound SSL
  12. Enable?Certificate is a Trust Anchor
  13. Save and exit all dialogs

This will add a valid issuer certificate that does not fail any constraints. If a higher level issuing certificate is not available then the disallowed algorithms and/or minimum key length must be adjusted

Modifying the security settings of the Gateway JDK

This process will disable the security requirements for?trusted certificates. This should only be done if a certificate of a higher certificate authority or a higher-security issuing certificate cannot be obtained from the applicable certificate authority. Perform the following procedure to disable the security requirements:

  1. Log in to the CA API Gateway as the?ssgconfig?user
  2. Select Option #3:?Use a privileged shell (root)
  3. Disable the configuration directive specifying allowed algorithms and key lengths:?sed -i '/^jdk.certpath/s/^/#/' /opt/SecureSpan/JDK/jre/lib/security/java.security
  4. Restart the API Gateway appliance

It may be desirable to lower the requirements instead of disabling them outright. If an administrator is aware of the allowed algorithms and/or lengths then the?jdk.certpath.disallowedAlgorithms line in /opt/SecureSpan/JDK/jre/lib/security/java.security can be changed to meet those organizational requirements. Please consult a security engineer for more information on the internal security requirements of your organization.

Disable the use of SunJSSE

The most drastic method of bypassing this behavior is to disable SunJSSE if it is currently in use. SunJSSE is not the default cryptographic suite used by the Gateway application. If SunJSSE is currently in use as the active cryptographic provider then it may have been enabled for a specific purpose. An administrator should ensure that documentation exists for why this property was changed and if that change can and should be reverted. If the change can be reverted safely then perform the following procedure in order to do so:

  1. Log in to the CA API Gateway as the?ssgconfig?user
  2. Select Option #3:?Use a privileged shell (root)
  3. Disable the use of SunJSSE: sed -i '/^com.l7tech.security.tlsProvider/s/^/#/' /opt/SecureSpan/Gateway/node/default/etc/conf/system.properties
  4. Restart the API gateway appliance
If this change is desired then it is strongly recommended that testing be performed in a non-production environment where SunJSSE is currently in use. Disabling the use of SunJSSE may have a negative impact on other changes to the Gateway that rely upon the use of SunJSSE.

Environment

Release:
Component: APIGTW
Powered by Wolken Software