VCF 9.1 Software Depot fails with error, "Failed to connect to the authorization server to obtain access token." when using a SSL Inspection proxy

Products

• VMware Cloud Foundation
• VCF Operations

Issue/Introduction

VCF Operations > Software Depot Wizard shows the following error:

Failed to connect to authorization server" or "Failed to connect to Broadcom OAuth Authorization server

 

 

A similar error is observed in the FDS pod log (depot-service):

javax.net.ssl.SSLHandshakeException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target

 

Environment

• VMware Cloud Foundation 9.1
• VCF Operations 9.1
• VCF Software Depot 9.1 

Cause

In network environments with SSL Inspection enabled (a security firewall or proxy performing TLS/HTTPS interception), the firewall intercepts outbound connections to online VCF software depot URLs (e.g. dl.broadcom.com, eapi.broadcom.com) and re-encrypts the traffic using its own CA-signed certificate instead of the real Broadcom certificate. Since the Software Depot trust store does not contain the firewall's Root CA/Intermediate CA, the TLS handshake fails and the online depot cannot be configured or used.

Resolution

Follow the below mentioned steps to  add the TLS certificates into the Fleet Depot Service's trust store:

 

1. Download the attached vmsp-utility.py script 
   
   
2. Identify a Control Plane Node from the VCF Operations UI.
   
   Build > Lifecyle > VCF Management > Components > VCF Services Runtime
   
   
   
   
3. Copy the vmsp-utility script to the Control Plane node's /home/vmware-system-user/ directory.
   
   
4. Copy the full certificate PEM file to the Control Plane node's /home/vmware-system-user/ directory
   
   
5. SSH into the Control Plane node with vmware-system-user (If password unknown follow steps in, "Resetting the vmware-system-user password for VMware Cloud Foundation Services Runtime")
   
   
6. Elevate to root with sudo -i 


7. Run the vmsp-utility.py script:
   
   

   root@vsp-6lq8s [ /home/vmware-system-user ]# python vmsp-utility.py
   ══════════════════════════════════════════════════════════════════
    VCF Management Service Utility
   ══════════════════════════════════════════════════════════════════
     Please ensure script is running on a Controller node
     vmware-system-user SSH password is required
   
     KUBECONFIG already set to /etc/kubernetes/admin.conf
   
   Resolving Node Host...
     NODE_HOST detected: https://192.168.1.213:5480
   
   Resolving VSP FQDN...
     VSP FQDN detected: vsp.vcf.lab
   
   Please enter the SSH password for vmware-system-user:
   
   Successfully generated token!
   
   Authenticating to VSP ([email protected])...
     VSP token generated.
     VSP component ID: 1d588015-dd49-4331-a181-017cc508592e
   
   1. Show proxy configuration
   2. Configure proxy
   3. Remove proxy
   4. Add certificates into truststore
   5. List certificates in truststore
   6. Quit
   
   Select one of the following options: 4
   
   ─── Add Certificates into Truststore ───
   
   Path to PEM chain file: /home/vmware-system-user/root.crt

8. Retry the Depot connection workflow.
   
   
9. The Software Depot UI will go blank. Wait for ~15 minutes for services to start.