Error: "Cannot delete addon harbor, since it is in deleting phase"

Products

• VMware Telco Cloud Automation

Issue/Introduction

• Harbor add-on stuck in processing
• Unable to manually delete Harbor due to error:
  Cannot delete addon harbor, since it is in deleting phase
  
• Under SHOW MORE, you see a Warning: Waiting for addon to get deleted with Reason: CAS110705
  
  

Environment

2.3

Cause

Kubernetes content in the affected namespace have finalizers remaining that need to be manually removed.

Resolution

1. SSH to the cluster Endpoint IP as capv

2. Identify the harbor tcaclusteraddon associated to the namespace associated to the workload cluster

   kubectl get tcaclusteraddons -n <namespace>

3. Edit tcaclusteraddons

   kubectl edit tcaclusteraddons <addonName> -n <namespace>

4. Delete the 2 lines associated to the finalizers

   e.g. 

   finalizers:
- machine.cluster.x-k8s.io

5. Delete workload cluster from the UI

6. Delete the VIM associated to the workload cluster from the UI

Remove namespace if stuck Terminating 

1. Describe the namespace

   kubectl describe ns <namespace>

   The NamespaceFinalizersRemaining section will detail the resources that remain and need to be removed manually. 

   Note: Focus on the following resources: machines, vspheremachine, and vspherevm 

   e.g
   Some content in the namespace has finalizers remaining: machine.cluster.x-k8s.io in 3 resource instances
Some content in the namespace has finalizers remaining: vspheremachine.infrastructure.cluster.x-k8s.io in 3 resource instances
Some content in the namespace has finalizers remaining: vspherevm.infrastructure.cluster.x-k8s.io in 3 resource instances

2. List all resource type found in the namespace, based on the finalizers detailed when describing the namespace:

   e.g. kubectl get machines -A -n <namespace>  

3. Edit the resource

   e.g. kubectl edit machines <machineName> -n <namespace>

4. Delete the 2 lines associated to the finalizers

   e.g. 

   finalizers:
- machine.cluster.x-k8s.io

5. Repeat steps 1-4 until the resources described in the namespace (machines, vspheremachine, and vspherevm) successfully terminates.