Software Depot Connected Mode Fails with OAuth Authentication Server Connection Error in VCF 9.1

Products

• VCF Operations

Issue/Introduction

When configuring or validating Connected Mode for the Software Depot in VMware Cloud Foundation (VCF) 9.1 behind an egress proxy performing SSL inspection, the operation fails in the SDDC Manager user interface. The user interface displays the following error message: 

Failed to connect to OAuth Authentication server

The issue occurs even after adding your proxy's CA certificate to the truststore of  software depot as per the KB VCF 9.1 Software Depot fails with error, "Failed to connect to the authorization server to obtain access token." when using a SSL Inspection proxy

The depot-service pod logs show the following TLS handshake failures:

ERROR [download_service, 0000000000000000, 0000] [Taskservice Impl, ds-exec-1] Exception happened while processing the binary with id depotconfig
com.vmware.evo.sddc.common.core.error.InternalServerErrorException: Failed to connect to Broadcom OAuth Authorization server.
at com.vmware.evo.sddc.lcm.tools.downloadservice.validator.DepotConnectionValidator.getAccessToken(DepotConnectionValidator.java:192)
Caused by: com.vmware.evo.sddc.lcm.depot_auth.exceptions.DepotAccessTokenIOException: org.bouncycastle.tls.TlsFatalAlert: insufficient_security (71)
at com.vmware.evo.sddc.lcm.depot_auth.DepotAccessTokenService.retrieveAccessToken(DepotAccessTokenService.java:107)

Caused by: org.bouncycastle.tls.TlsFatalAlert: insufficient_security (71)
at org.bouncycastle.tls.TlsDHUtils.receiveDHConfig(Unknown Source)

Executing a verbose curl diagnostic through the customer proxy targeting [https://eapi.broadcom.com](https://eapi.broadcom.com) confirms intermediate certificate interception:

curl -vvv -x http://<PROXY_FQDN>:<PROXY_PORT> https://eapi.broadcom.com
...
* Server certificate:
*   subject: C=US; ST=California; L=Palo Alto; O=Broadcom Inc.; CN=eapi.broadcom.com
*   issuer: PROXY CERTIFICATE_REDACTED
* OpenSSL verify result: 13
* SSL certificate OpenSSL verify result: self-signed certificate in certificate chain (19)
curl: (60) SSL certificate OpenSSL verify result: self-signed certificate in certificate chain (19)

Environment

VMware Cloud Foundation 9.1

Cause

The egress proxy intercepts HTTPS traffic and re-signs certificates. When the proxy offers a weak 1024-bit DHE configuration, the strict BouncyCastle TLS engine rejects the connection.

Resolution

This issue is targeted to be fixed in an upcoming release of VMware Cloud Foundation.

Note: For further updates on the fix status, subscribe to this article.

Workaround:

Perform the following steps on the network egress proxy to bypass SSL inspection and ensure proper cryptographic passthrough:

1. Configure an SSL Inspection Exemption or SSL Bypass rule on the egress proxy server for the following Broadcom endpoints:

   1. https://eapi.broadcom.com
   2. https://dl.broadcom.com
   3. https://vvs.broadcom.com

2. Verify the exemption by running:

   curl -vvv -x http://<proxy_fqdn>:<port> https://eapi.broadcom.com

3. Confirm the server certificate issuer presents the Broadcom public CA instead of the local proxy certificate. Once verified, re-validate the Connected Mode configuration in the VCF 9.1 Software Depot UI.

Additional Information

For more information on proxy configuration, see Configure a Proxy Server.