SDDC Manager fails to connect to online depot with error "Updating depot user credentials failed. Depot Not Available"

Products

• VMware SDDC Manager

Issue/Introduction

• SDDC Manager is unable to connect to depot to download updates from the internet facing VMware by Broadcom public repositories.
  
  
• When attempting to connect to depot, the below error is seen.
  
  
  
  
  
• The below snippets are seen in SDDC LCM logs:

/var/log/vmware/vcf/lcm/lcm-debug.log

YYYY-MM-DD T HH:MM:SS DEBUG [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.d.utils.CookieUtils,http-nio-127.0.0.1-7400-exec-7] VCF_DEPOT Depot Http Cookies: []
YYYY-MM-DD T HH:MM:SS DEBUG [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.depot.DepotDownloader,http-nio-127.0.0.1-7400-exec-7] Getting file size for [/COMP/SDDC_MANAGER_VCF/index.v3] from URL[https://dl.broadcom.com:443/<downloadToken>/PROD/COMP/SDDC_MANAGER_VCF/index.v3]
YYYY-MM-DD T HH:MM:SS DEBUG [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.d.utils.CookieUtils,http-nio-127.0.0.1-7400-exec-7] VCF_DEPOT Depot Http Cookies: []
YYYY-MM-DD T HH:MM:SS DEBUG [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.depot.DepotDownloader,http-nio-127.0.0.1-7400-exec-7] Executing HEAD https://dl.broadcom.com:443/<downloadToken>/PROD/COMP/SDDC_MANAGER_VCF/index.v3 HTTP/1.1
YYYY-MM-DD T HH:MM:SS DEBUG [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.depot.DepotDownloader,http-nio-127.0.0.1-7400-exec-7] Got response: HTTP/1.1 400 Bad Request
YYYY-MM-DD T HH:MM:SS ERROR [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.depot.DepotDownloader,http-nio-127.0.0.1-7400-exec-7] Error getting file size, got response: HTTP/1.1 400 Bad Request
YYYY-MM-DD T HH:MM:SS ERROR [vcf_lcm,819a0e70d17c4f97,1101,[email protected]] [c.v.e.s.l.b.d.depot.DepotDownloader,http-nio-127.0.0.1-7400-exec-7] Got Http error[400] while downloading manifest index [/COMP/SDDC_MANAGER_VCF/index.v3]
YYYY-MM-DD T HH:MM:SS ERROR [vcf_lcm,819a0e70d17c4f97,1101] [c.v.e.s.l.a.r.c.d.DepotUserCredentialController,http-nio-127.0.0.1-7400-exec-7] Failed to update User Credential
com.vmware.evo.sddc.lcm.model.depot.exception.DepotConnectionFailureException: Error downloading manifest [/COMP/SDDC_MANAGER_VCF/index.v3] from dl.broadcom.com:443

• Validated the URL connectivity via curl
  
  curl -v https://dl.broadcom.com:443/<downloadToken>/PROD/COMP/SDDC_MANAGER_VCF/index.v3
  
  
• Checked SDDC manager UI and the application-prod.properties file in /opt/vmware/vcf/lcm/lcm-app/conf/ and confirmed that proxy is not in use.

Environment

SDDC Manager Version 4.5.1

SDDC Manager Version 4.5.2

Cause

The issue was due to the proxy server being enabled in the platform database on the SDDC manager, even though it is disabled in the UI and does not reflect in the application-prod.properties file.

Resolution

Note: Take a valid snapshot of the SDDC manager VM, and execute the steps below:

1. Login as the root user to the SDDC manager via SSH.
   
   
2. Query the proxy information from the platform DB 
   
      psql -h localhost -U postgres -d platform -c "select * from proxy_config"
      
      If the proxy_enabled section shows "t", then proxy is enabled in the database.
      
3. Disable proxy from the platform database 
   
      psql -h localhost -U postgres -d platform -c "update proxy_config set proxy_enabled='f' where id='Enter_ID_from_step_2'"
      
4. Restart all SDDC manager services
   
      /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh

Once the above changes are done, from the SDDC manager UI, login to the depot and check the connectivity.