How to Identify if a vSphere Kubernetes Cluster is Undergoing Migration from vSphere 7.X to vSphere 8.X

Products

• VMware vSphere 7.0 with Tanzu
• VMware vSphere with Tanzu
• vSphere with Tanzu
• Tanzu Kubernetes Runtime

Issue/Introduction

This KB is intended to identify if a vSphere Kubernetes Cluster, also known as a Tanzu Kubernetes Cluster (TKC), is undergoing migration.

It will also advise against certain actions such as performing changes or upgrades to the migrating cluster.

Migration of all vSphere Kubernetes Clusters in a Supervisor cluster occurs after initiating an upgrade of the environment from vSphere 7.X to vSphere 8.X.

Environment

Environment that is upgrading from vSphere 7.X to vSphere 8.X

This issue can occur regardless of whether the affected migrating cluster is managed by Tanzu Mission Control (TMC)

Cause

Upgrades from vSphere 7 to vSphere 8 undergo an automatic migration of wcp objects into vsphere objects, and virtualmachineimages into clustervirtualmachineimages (cvmi).

The associated components are updated to reference the newly created objects for migration.

Documentation: "Upgrading from any vCenter Server release to any vCenter Server 8.x release"

Resolution

Upgrades from vSphere 7 to vSphere 8 undergo an automatic migration of wcp objects into vsphere objects and virtualmachineimages into clustervirtualmachineimages (cvmi).

 

IMPORTANT: While a cluster is migrating, it is important to not make any changes to the cluster or perform deletions of migrating objects:

• Editing a migrating cluster can cause further issues during the migration and cause the migration of the cluster to become stuck.
  
  
• It is advised to not make any configuration changes to the migrating cluster. This can prevent the cluster's migration from completing.
  
  
• Please do not start an upgrade of the migrating cluster. This will cause the cluster's migration to become stuck.
  
  
• Do not perform deletions of objects to force the migration. This will lead to permanent loss of those migrating objects.

 

Please see the below for steps on how to identify a cluster that has not yet finished its migration.

1. Connect to the Supervisor cluster context
2. Identify all vSphere Kubernetes clusters with the migrate-tkc label:

   • kubectl get tkc -o yaml -n <cluster namespace> | grep "migrate-tkc"
     
     

3. Check regarding the migrating cluster's progress of wcp object migration to vsphere object equivalents:

   • kubectl get wcpcluster,wcpmachine,wcpmachinetemplate -n <cluster namespace>

   • kubectl get vspherecluster,vspheremachine,vspheremachinetemplate -n <cluster namespace>

     • NOTE: wcp objects being present may indicate that the migration is yet to complete for the corresponding vSphere Kubernetes cluster(s).
     • However, these wcp objects may still be present even if the corresponding TKC object does not have the "migrate-tkc" label.
       • This may indicate an issue with the system properly cleaning up the leftover wcp objects.
     • It would be best to reach out to VMware by Broadcom Technical Support referencing this KB article for help in cleaning up any leftover migration objects.
       • This is to avoid deletion of objects that are still in use or not yet fully migrated.
         
         
4. Confirm on the status of the migration of virtualmachineimage migration to clustervirtualmachineimage (cvmi):

   • kubectl get virtualmachineimages -A

   • kubectl get cvmi -A