"Got Http error[403] while downloading bundle", OBTU tool fails to download bundles from the online repositories

Products

• VMware Cloud Foundation 4.x
• VMware Cloud Foundation 5.x

Issue/Introduction

VCF Offline Bundle Transfer Utility (OBTU) Tool bundle downloads from internet facing VMware by Broadcom public repositories fails with below errors . 

Got Http error[403] while downloading bundle [/evo/vmw/bundles/bundle-124938.tar]
Failed downloading bundle: bundle-124938.. Will retry after 5 seconds

Environment

VMware Cloud Foundation (OBTU tool)

Cause

Public facing repository URLs and authentication mechanisms are changing. Download URLs are no longer common but unique for each customer therefore will require to be re-configured.

Resolution

1. Download the Token from Broadcom Support Portal by following the steps in VCF Authenticated downloads Configuration Update Instructions. If you already have it, apply the same by following below instructions.

2. Download Bundles with Bundle Transfer Utility

   1. Download and setup OBTU following this document Download Bundles with the Bundle Transfer Utility
   2. Edit the file and change the following property values as below : /opt/vmware/vcf/lcm/lcm-tools/conf/application-prod.properties

lcm.depot.adapter.host=dl.broadcom.com
lcm.depot.adapter.remote.rootDir=/<downloadToken>/PROD
lcm.depot.adapter.remote.repoDir=/COMP/SDDC_MANAGER_VCF
lcm.depot.adapter.remote.lcmManifestDir=/COMP/SDDC_MANAGER_VCF/lcm/manifest
lcm.depot.adapter.remote.bundletransferconfig.repoDir:/COMP/SDDC_MANAGER_VCF/obtu

Note: Replace <downloadToken> with the actual downloaded token id.
      
      
   3. Follow the remaining setup to download bundles and upload to SDDC-Manager.
      
      

3. Setup offline depot.

   1. Follow VCF Offline Depot deployment
   2. Before running the command with -setupOfflineDepot configure application-prod.properties following the steps above.
   3. Run the command to download the offline depot contents.
   4. The directory structure of the downloaded files will look like this
      

      └── <download Token>

          └── PROD

              ├── COMP

              │   └── SDDC_MANAGER_VCF

              │   ├── bundles

              │   │   └── bundle-202281.tar

              │   ├── Compatibility

              │   │   └── VmwareCompatibilityData.json

              │   ├── deltaFileDownloaded

              │   ├── deltaFileDownloaded.md5

              │   ├── index.v3

              │   ├── lcm

              │   │   └── manifest

              │   │   └── v1

              │   │       └── lcmManifest.json

              │   ├── manifests

              │   │   ├── bundle-202281.manifest

              │   │   └── bundle-202281.manifest.sig

              │   └── tmp

              │       ├── index.v3

              │       └── lcmManifest.json

              └── vsan

                  └── hcl

                      ├── all.json

                      └── lastupdatedtime.json

   5. If you already have files that are downloaded to the old structure, you can either re-download them from the online depot or copy them into the new folder structure as follows from the --offlineDepotRootDir
      1. cp -r PROD2/evo/vmw/bundles/* <download Token>/PROD/COMP/SDDC_MANAGER_VCF/bundles/
      2. cp -r PROD2/evo/vmw/manifests/* <download Token>/PROD/COMP/SDDC_MANAGER_VCF/manifests/
      3. Merge the deltaFileDownloaded deltaFileUploaded from both the directory into the  <download Token>/PROD/COMP/SDDC_MANAGER_VCF/ directory
         • cat <download Token>/PROD/COMP/SDDC_MANAGER_VCF/deltaFileDownloaded PROD2/evo/vmw/deltaFileDownloaded | sort | uniq > deltaFileDownloaded
         • cp deltaFileDownloaded <download Token>/PROD/COMP/SDDC_MANAGER_VCF/
      4. Calculate the md.5 checksum again for the deltaFileDownloaded
         • cd <download Token>/PROD/COMP/SDDC_MANAGER_VCF/
         • md5sum deltaFileDownloaded | cut -d ' ' -f 1  > deltaFileDownloaded.md5
   6. Configure the SDDC Manager depot: This can be done by running the script described in the KB One VCF Depot Reconfiguration script.
   7. Setup offline depot in SDDC Manager
      1. Open SDDC manager UI
      2. Navigate to “Lifecycle Management" -> “Depot Settings” -> “Set up an Offline Depot”
      3. Enter the offline depo hostname, port, username and password
   8. OBTU will not support async patch download for offline depot, use AP Tool for this.
   9.