TKGi MC (Management Console) upgrade from TKGi 1.17.6 to 1.18.5 - BOSH Director reporting 'Missing stemcell' and error advising required_stemcell_version 1.486 and required_stemcell_os ubuntu-jammy.
search cancel

TKGi MC (Management Console) upgrade from TKGi 1.17.6 to 1.18.5 - BOSH Director reporting 'Missing stemcell' and error advising required_stemcell_version 1.486 and required_stemcell_os ubuntu-jammy.

book

Article ID: 379064

calendar_today

Updated On:

Products

VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

In the TKGi MC (Management Console) after upgrading from TKGI 1.17.6 to TKGi 1.18.5 the below errors are observed - where BOSH director advises that there is a 'Missing Stemcell' and the upgrade fails with below error advising required_stemcell_version 1.486 and required_stemcell_os ubuntu-jammy.



Perform network configuration SuccessCompleted
 
Deploy ops-manager 3.0.31+LTS-T SuccessCompleted
 
Generate and register certificates SuccessCompleted
 
Deploy bosh SuccessCompleted
 
Deploy pivotal-container-service 1.18.5-build.10 FailedFailed step 5/8: Deploying TKGI, installation failed to trigger: request failed: unexpected response: HTTP/1.1 422 Unprocessable Content Content-Length: 408 Cache-Control: private, no-store Connection: keep-alive Content-Security-Policy: script-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-inline'; object-src 'none'; Content-Type: application/json; charset=utf-8 Date: Thu, 19 Sep 2024 09:59:17 GMT Expires: Fri, 01 Jan 1990 00:00:00 GMT Pragma: no-cache Referrer-Policy: strict-origin-when-cross-origin Server: Ops Manager Strict-Transport-Security: max-age=63072000; includeSubDomains X-Content-Type-Options: nosniff X-Download-Options: noopen X-Frame-Options: SAMEORIGIN X-Permitted-Cross-Domain-Policies: none X-Request-Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx X-Runtime: 7.503894 {"errors":["Installation is incomplete. One or more tiles require configuration."],"deployment_errors":{"products":[{"identifier":"p-bosh-xxxxxxxxxxxxxxxxxxxx","complete":false,"network":{"assigned":true},"availability_zone":{"assigned":true},"stemcells":[{"assigned":true,"required_stemcell_version":"1.486","required_stemcell_os":"ubuntu-jammy"}],"properties":[],"resources":{"jobs":[]},"verifiers":[]}]}}: installation failed to trigger: request failed: unexpected response: HTTP/1.1 422 Unprocessable Content Content-Length: 408 Cache-Control: private, no-store Connection: keep-alive Content-Security-Policy: script-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-inline'; object-src 'none'; Content-Type: application/json; charset=utf-8 Date: Thu, 19 May 2024 10:00:05 GMT Expires: Fri, 01 Jan 1990 00:00:00 GMT Pragma: no-cache Referrer-Policy: strict-origin-when-cross-origin Server: Ops Manager Strict-Transport-Security: max-age=63072000; includeSubDomains X-Content-Type-Options: nosniff X-Download-Options: noopen X-Frame-Options: SAMEORIGIN X-Permitted-Cross-Domain-Policies: none X-Request-Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx X-Runtime: 7.503894 {"errors":["Installation is incomplete. One or more tiles require configuration."],"deployment_errors":{"products":[{"identifier":"p-bosh-xxxxxxxxxxxxxxxxxxxx","complete":false,"network":{"assigned":true},"availability_zone":{"assigned":true},"stemcells":[{"assigned":true,"required_stemcell_version":"1.486","required_stemcell_os":"ubuntu-jammy"}],"properties":[],"resources":{"jobs":[]},"verifiers":[]}]}}

Environment

TKGi 1.17.6 to 1.18.5
BOSH 3.0.31+LTS-T

Cause

The TKGi MC for 1.18.5 comes preload with base stemcell 1.486ubuntu-jammy

Resolution

      1. SSH to 1.18.5 TKGI MC VM server( on either upgrade failed TKGi MC VM or on the newly deploy TKGi MC VM 1.18.5)
      2. Stop TKGi API server by the following command:

        systemctl stop pks-mgmt-server.service


      3. Backup DB and artifacts.yaml

        mkdir /root/backup
        
        cp /storage/data/artifacts/artifacts.yaml /root/backup/
        
        cp /storage/data/pks.db /root/backup/
      4. Download 1.506 stemcell file to the folder /storage/data/artifacts/stemcells-ubuntu-jammy/1.506

        mkdir /storage/data/artifacts/stemcells-ubuntu-jammy/1.506
        
        cd /storage/data/artifacts/stemcells-ubuntu-jammy/1.506
        
        wget <download_url>


      5. Update the version and path of pks-stemcell and harbor-stemcell to 1.506 in the /storage/data/artifacts/artifacts.yaml

        vi /storage/data/artifacts/artifacts.yaml
      6. Replace the pks-stemcell and harbor-stemcell section with the following section:

        pks-stemcell:   name: pks-stemcell  version: "1.506"
        path: /storage/data/artifacts/stemcells-ubuntu-jammy/1.506/bosh-stemcell-1.506-vsphere-esxi-ubuntu-jammy-go_agent.tgz
        harbor-stemcell:   name: harbor-stemcell  version: "1.506"
        path: /storage/data/artifacts/stemcells-ubuntu-jammy/1.506/bosh-stemcell-1.506-vsphere-esxi-ubuntu-jammy-go_agent.tgz
      7. Start TKGI api server by following command

        systemctl start pks-mgmt-server.service
      8. Open 1.18.5 TKGI MC UI.

        8.1 continue to your upgrade as usual(If you deployed a new TKGI MC VM)

        or

        8.2 continue to your upgrade by applying the upgrade configuration again from UI(from TKGi MC VM that has failed)