Failure attaching foundation to Tanzu Hub in air-gapped environment: "stemcells-ubuntu-jammy-1.1065 has unexpected status FAILED""

Products

• VMware Tanzu Platform Core
• VMware Tanzu Platform - Hub

Issue/Introduction

• Tanzu Hub 10.4 installed in an air-gapped environment might encounter errors when attaching foundations.
• This occurs after using cf CLI to manually download the necessary product capabilities as detailed in the Tanzu cf CLI commands for air-gapped environments documentation.
  • This might follow the manual corrective action prescribed in this KB.
• This failure occurs on foundations that use Stemcells on versions lower than 1.1065.
• When attaching foundations that utilize Stemcells on versions lower than 1.1065, the error: "Capabilities download workflow warehouse-release-download-################################-stemcells-ubuntu-jammy-1.1065 has unexpected status FAILED" is returned during the attach workflow in Tanzu HUB GUI.

Environment

Tanzu Hub 10.4 air-gapped.

Cause

This occurs because the cf download-content command downloads the files with the display name, but the attach workflow expects the Stemcell file to be file name. This leads to a mismatch and the reported failure during the attach workflow. 

Resolution

This is resolved in the first patch release of Tanzu Hub: 10.4.1 with the ability to use Proxy for automatic product-warehouse download.

 

Workaround

To work around this condition in environments where the Stemcell has already been downloaded, you have to rename the location reference after download in the download-content-result.yml file as well as the actual downloaded file, then delete the incorrectly named file from Hub and re-upload with the correct name. 

 

Modify the downloaded product in preparation for upload

1. 1. On the Jumpbox used to download the Stemcell as detailed in the Tanzu cf CLI commands for air-gapped environments documentation, navigate to the folder used to download the product into.
   2. Backup the download-content-result file and edit it:
      
      # cp download-content-result.yml old_download-content-result.yml
# vi download-content-result.yml
      
      
      • Modify the location line to:
        
        products/stemcells-ubuntu-jammy/1.1065/bosh-stemcell-1.1065-vsphere-esxi-ubuntu-jammy-go_agent.tgz
        
        
   3. Change the name of the file:
      
      products/stemcells-ubuntu-jammy/1.1065/'Ubuntu Jammy Stemcell for vSphere 1.1065'
      
      to:
      
      products/stemcells-ubuntu-jammy/1.1065/bosh-stemcell-1.1065-vsphere-esxi-ubuntu-jammy-go_agent.tgz
      
      
      

Remove the incorrectly named Stemcell file from Hub using the GraphQL Altair interface

1. 1. Access Hub GraphQL API on web browser: https://<HUB_FQDN>/hub/altair (use this KB on how to access if needed).
   2. After entering the Raw Token from the Hub GUI, use the following two queries to gather the document ID (this can be found in the download-content-result.yml file you used to manually download the Stemcell), then use the resulting document ID in the delete query to delete the document:
      
      query QueryDocument {
    documentQuery {
        queryDocuments(
            category: BUILD_ARTIFACTS
            entityIds: "vrn/provider:Tanzu/origin:Platform/ReleaseFile:<FILE_ID_FROM_download-content-result.yml>"
          ) {
            count
            totalCount
            documents {
                id
                name
                state
                type
            }
        }
    }
}


mutation deleteDocument {
  documentMutation {
    deleteDocuments(documentIds: "<DOCUMENT_ID_FROM_FIRST_QueryDocument_QUERY>")
  }
}



   3. Example for Stemcell that has file ID 231481 in the download-content-result.yml file:
      
      
      • QueryDocument query gathering the ID of the document in preparation for deletion:
        
        
        
        
      • deleteDocument mutation query deleting the document using ID recovered in the QueryDocument query:
        
        
        
        
        
      • download-content-result.yml contents example:
        
        downloads:
- slug: stemcells-ubuntu-jammy
  result: SUCCESS
  releases:
  - version: "1.1065"
    id: 540267
    files:
    - id: 231481
      location: products/stemcells-ubuntu-jammy/1.1065/bosh-stemcell-1.1065-vsphere-esxi-ubuntu-jammy-go_agent.tgz
      checksumSha256: 8752a0d2458c6eeff41ad999d8574cd580dffbblddd204cb00529b3cb5489fde
      hubDocumentServiceTtlSeconds: 2628000
      hubDocumentServiceDocumentCategory: BUILD_ARTIFACTS

 

Re-upload the Stemcell

# ./cf upload-content --metadata download-content-result.yml

 

 

These steps will allow the Stemcell download step to be skipped during the foundations attach workflow in Tanzu Hub specifically in air-gapped environments.