"A general system error occurred: Failed to download VIB(s): Error: HTTP Error Code: 403", vLCM fails to download the ESXi patches and images from online repositories

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• vCenter Lifecycle Management (vLCM) fails to download ESXi patches/images from VMware by Broadcom public repositories with following error message:

A general system error occurred:
Failed to download VIB(s): URL:
https://hostupdate.broadcom.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/<vib_name>.vib
Error: HTTP Error Code: 403

Connectivity Status under Lifecycle Manager -> Settings -> Patch Setup for hostupdate.vmware.com URL's show Not Connected or Validating.


• While patching Esxi using vCenter Server UI fails when download repository is configured as https://hostupdate.vmware.com/software/VUM/PRODUCTION/<>/vmw-depot-index.xml

"Cannot download VIB: ''. This might be because of network issues or the specified VIB does NOT exist or does NOT have a proper 'read' privilege set. Please make sure the specified VIB exists and is accessible from vCenter Server."

• /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log

YYYY-MM-DDTHH:MM:SS info vmware-vum-server[1069416] [Originator@6876 sub=httpDownload] [httpDownloadPosix 769] curl_easy_perform() succeeded - url: https://hostupdate.vmware.com/software/VUM/PRODUCTION/vmtools-main/vmw-depot-index.xml
YYYY-MM-DDTHH:MM:SS error vmware-vum-server[1069416] [Originator@6876 sub=DownloadMgr] [downloadMgr 705] Executing download job {140344436625712} throws error: HTTP Error Code: 403
YYYY-MM-DDTHH:MM:SS info vmware-vum-server[1069416] [Originator@6876 sub=DownloadMgr] [downloadMgr 807] Download failed but destination file /tmp/vciHB8meY exists and is valid. Ignoring error
...
YYYY-MM-DDTHH:MM:SS error vmware-vum-server[1069421] [Originator@6876 sub=Default] [updateDownloaderImpl 116] File download error: Downloading file: https://hostupdate.vmware.com/software/VUM/PRODUCTION/vmtools-main/vmw-depot-index.xml failed, 0 byte downloaded.

Cause

Public facing repository URLs and authentication mechanisms are changing. Download URLs are no longer common but unique for each customer therefore will require to be re-configured.

Resolution

1. Download Token from Broadcom Support Portal by following the steps in VCF Authenticated downloads Configuration Update Instructions. If you already have it, apply the same by following below instructions.
   
   
2. Disable the default repositories and configure the repositories with the Token
   
   Note: Scripted method to update the URLs are available in Authenticated Download Configuration Update Script.
   
   
   1. Log into vSphere client
   2. On the vSphere Client Home page, Navigate to Lifecycle Manager.
   3. Click 'settings' tab. Under 'settings' tab,
   4. Go to Administration → Patch Setup.
   5. Disable default vLCM URLs by clicking "Disable" button:
      
      Note: If the vCenter Server is running on 7.0 U3c (build 19234570) or 8.0 GA versions (8.0GA, 8.0a, 8.0b & 8.0c), configuring the Custom URLs mentioned below will fail. Please perform Update Manager DB reset before proceeding with below steps if the vCenter Server is on these impacted versions. 
      
      
      
      
   6. Add the below URLs one by one by clicking 'NEW':
      
      
      
      New URLs:
      
      https://dl.broadcom.com/<Download Token>/PROD/COMP/ESX_HOST/main/vmw-depot-index.xml
https://dl.broadcom.com/<Download Token>/PROD/COMP/ESX_HOST/addon-main/vmw-depot-index.xml
https://dl.broadcom.com/<Download Token>/PROD/COMP/ESX_HOST/iovp-main/vmw-depot-index.xml
https://dl.broadcom.com/<Download Token>/PROD/COMP/ESX_HOST/vmtools-main/vmw-depot-index.xml
      
      Note: Replace <Download Token> with the actual downloaded token id.
      
      
   7. Restart update manager service from an SSH session to vCenter Server
      
      service-control --restart vmware-updatemgr
      
      
   8. Download the content by going to vSphere Client Home page → Lifecycle Manager and selecting "ACTIONS" → Updates → Sync updates.
   9. Follow the steps from Error: "The vCenter Server is not able to reach the specified URL" vCenter Server patching via VAMI fails to download the updates from online repositories
       to update VAMI URL 

Additional Information

If you are encountering this issue even if your vCenter Server is not running one of the above listed versions or even in spite of adding the "Download Token" (and updating the URLs) as per the Resolution section, please perform Update Manager DB reset and retry the instructions mentioned in the Resolution section.