TKG 2.1 upgrade to TKG 2.2 fails when searching for 1.25.7 Tanzu Kubernetes Release on Air Gapped environment
search cancel

TKG 2.1 upgrade to TKG 2.2 fails when searching for 1.25.7 Tanzu Kubernetes Release on Air Gapped environment

book

Article ID: 326387

calendar_today

Updated On:

Products

VMware

Issue/Introduction

Symptoms:

- When upgrading TKGm 2.1 to TKG 2.2 the procedure fails with an error:
could not resolve TKR/OSImage for controlPlane, machineDeployments: [md-0 md-1 md-2], query: {controlPlane: {k8sVersionPrefix: 'v1.25.7+vmware.2-tkg.1', tkrSelector: '', osImageSelector: 'image-type=ova,os-name=ubuntu'}, machineDeployments: [{k8sVersionPrefix: 'v1.25.7+vmware.2-tkg.1', tkrSelector: '', osImageSelector: 'image-type=ova,os-name=ubuntu'} {k8sVersionPrefix: 'v1.25.7+vmware.2-tkg.1', tkrSelector: '', osImageSelector: 'image-type=ova,os-name=ubuntu'} {k8sVersionPrefix: 'v1.25.7+vmware.2-tkg.1', tkrSelector: '', osImageSelector: 'image-type=ova,os-name=ubuntu'}]}, result: {controlPlane: {k8sVersion: '', tkrName: '', osImagesByTKR: map[]}, machineDeployments: [{k8sVersion: '', tkrName: '', osImagesByTKR: map[]} {k8sVersion: '', tkrName: '', osImagesByTKR: map[]} {k8sVersion: '', tkrName: '', osImagesByTKR: map[]}]}", "UID": "4b7051a0-f655-45c9-b1f1-336bbae5da54", "allowed": false}

- Running kubectl get tkr does not list the new TKRs for TKGm 2.2, and the upgrade cannot proceed.

Environment

VMware Tanzu Kubernetes Grid 2.1.1
VMware Tanzu Kubernetes Grid 2.4.0
VMware Tanzu Kubernetes Grid 2.3.0

Cause


The tkr-source-controller do not find the tkr-vsphere-nonparavirt-v1.25.7_vmware.2-tkg.1 and tkg-vsphere-repository-nonparavir-1.25.7_vmware.2-tkg.1 package on Harbor

Resolution


Upload the missing packages manually using imgpkg to Harbor.

Workaround:

Follow these steps when uploading the bundle to the private Harbor (as stated in the  Prepare an Internet-Restricted Environment page):
 
  1. Check if the tkr-vsphere-nonparavirt-v1.25.7_vmware.2-tkg.1 and tkg-vsphere-repository-nonparavir-1.25.7_vmware.2-tkg.1 packages exist in the private Harbor repository.
  2. Follow the air gap procedure to download the bundle locally using tanzu isolated-cluster download-bundle --tkg-version v2.2.0 command.
  3. Upload the bundle using tanzu isolated-cluster upload-bundle --destination-repo <private-harbor> to your private Harbor.
  4.  Copy the leftover packages manually using imgpkg:
    1. imgpkg copy --tar ${source_dir}/tkr-vsphere-nonparavirt-v1.25.7_vmware.2-tkg.1.tar --to-repo <remote-harbor-url>/tkr-vsphere-nonparavirt --registry-ca-cert-path ${harbor-cert-path}/ca.crt
    2. imgpkg copy --tar ${source_dir}/tkr-repository-vsphere-nonparavirt-v1.25.7_vmware.2.tar --to-repo <remote-harbor-url>/tkr-repository-vsphere-nonparavirt --registry-ca-cert-path ${harbor-cert-path}/ca.crt


Additional Information

https://kb.vmware.com/s/article/92524 - How to Increase tkr-source-controller deployment resource limits on TKG 2.1.x to resolve 'OOMKilled' Error