Download patch bundle with Async Patch Tool failed with "unable to find valid certification path to requested target"

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• Download patch bundle with Async Patch Tool in SDDC Manager failed with below messages:
  
  [main] DEBUG [com.vmware.vcf.secure.truststore.DynamicTrustManager]
                      Error checking certificate chain <unexpected_certificate_for_dl.broadcom.com> for validity.
xxxxx.security.validator.ValidatorException: PKIX path building failed: xxxx.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
  
  
• SDDC Manager connection to depot website dl.broadcom.com goes through intermediate network device, such as firewall. 
• The depot website certificate has been modified by intermediate network device which can be verified with below command in SDDC Manager:
  
  openssl s_client -connect dl.broadcom.com:443
  
  The expected subject and issuer of depot website dl.broadcom.com:
  
  subject=CN = 0bxxxxxx.xxxx.xxxxxx.com
issuer=C = US, O = Google Trust Services, CN = WE1
  
  If the subject and issuer are different with above, means that the depot website certificate has been tampered. 

Environment

VMware Cloud Foundation 5.x

Resolution

To resolve the issue:

1. Get the trusted root certificate of depot website which has been modified by intermediate network device with the commands:
   
   openssl s_client -connect dl.broadcom.com:443 -showcerts
   
   

2. With the release of VCF 9.x, the vSAN HCL and other files are stored on a storage.googleapis.com endpoint, and requires that site to be open as a destination from your VM running the OBTU.

   openssl s_client -connect storage.googleapis.com:443 -showcerts

3. Save the trusted root certificate to a .crt format with the content:
   
   -----BEGIN CERTIFICATE-----
.....
-----END CERTIFICATE-----
4. Add the trusted root certificate to the Trusted Store of Async Patch Tool:
   
   /opt/obtu/jre/lin64/bin/keytool -importcert -file trusted-root.crt -keystore <path-aptool-extracted>/jre/lin64/lib/security/cacerts
   
   
5. Set the JRE environment to Async Patch Tool JRE:
   
   export CUSTOM_JAVA_PATH=<path-aptool-extracted>/jre/lin64
   
   
6. Run the Async Patch Tool to down patch bundle again.

Additional Information

Attempting offline bundle downloads using the LCM Bundle Transfer Utility fails with username or password is invalid for user