VCF 9.1 Online Depot setup fails with VCF_ERROR_INTERNAL_SERVER_ERR

Products

• VMware SDDC Manager / VCF Installer
• VMware Cloud Foundation

Issue/Introduction

• Configuring the Online Depot in VCF Operations 9.x or SDDC Manager 9.x fails with the following error in the UI:
  
  A problem has occurred on the server. Reference token <TOKEN>

Error setting Online depot configuration

org.bouncycastle.tls.TlsFatalAlert: certificate_unknown(46)

certificate_unknown(46)

Unable to construct a valid chain

Unable to find certificate chain.

 

• The /var/log/vmware/vcf/lcm/lcm-debug.log on the SDDC Manager shows entries similar to:
  
  ERROR [vcf_lcm,...] [c.v.v.l.r.a.c.v.s.DepotSettingsController] Update Depot Settings com.vmware.evo.sddc.lcm.depot_auth.exceptions.DepotAccessTokenIOException: java.net.SocketException: Network is unreachableERROR [vcf_lcm,...] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler] [<TOKEN>] VCF_ERROR_INTERNAL_SERVER_ERROR A problem has occurred on the server.

 

• When testing connectivity manually, the curl output confirms a certificate trust issue:
  
  # curl -vvv https://eapi.broadcom.com/vcf/generateToken -k

ERROR: * OpenSSL verify result: 13* SSL certificate verification failed, continuing anyway!* [SSL] ossl_connect, done

 

Environment

VCF 9.x

Cause

• This issue occurs when the SDDC Manager appliance cannot establish a secure connection to the Broadcom endpoints required for downloading binaries (eapi.broadcom.com, dl.broadcom.com, vvs.broadcom.com, vsanhealth.vmware.com).
• The most common cause is an intercepting proxy using an internal Certificate Authority (CA) that is not trusted by the appliance's Java trust store.

Resolution

Step 1: Verify Connectivity

Run the following command from the SDDC Manager CLI to identify if the SSL handshake fails without the -k (insecure) flag:

curl -vvv https://eapi.broadcom.com/vcf/generateToken

If the output results in SSL certificate problem, the certificate chain must be trusted.

Step 2: Import Proxy/Root Certificate

1. Log in to SDDC Manager as vcf and elevate to root.
   
   
2. Run the following command to import the certificate to the Java keystore:
   
   echo off | openssl s_client -proxy [PROXY_IP]:[PROXY_PORT] -connect eapi.broadcom.com:443 2>&1 | openssl x509 -outform PEM 1> /tmp/depot.crt && keytool -importcert -alias broadcom_eapi -file /tmp/depot.crt -keystore /etc/alternatives/jre/lib/security/cacerts --storepass changeit

Note: Replace [PROXY_IP] and [PROXY_PORT] with the environment's proxy details.

Step 3: Whitelist Endpoints

Ensure the following FQDNs are whitelisted on the firewall/proxy and excluded from SSL decryption/inspection:

• dl.broadcom.com
• eapi.broadcom.com
• vvs.broadcom.com
• vsanhealth.vmware.com

Step 4: Configure Depot in UI

1. Navigate to Fleet Management > Lifecycle > VCF Management > Depot Configuration.
   
   
2. Re-enter the Download Token in the Online Depot settings.
   
   
3. Verify the tile displays Depot connection is active.

Additional Information

How to import Proxy server certificate to SDDC manager trust store
Troubleshooting VCF Depot Connection Issues
Public URLs Required for Online Functionalities for VCF and vSphere Foundation