Deleting all CRDs from the Management and Workload clusters

Products

• VMware Telco Cloud Automation

Issue/Introduction

All CRDs inside the Kubernetes workload cluster, including addon CRDs , were deleted and no etcd backup is available.

Environment

2.x , 3.x

Cause

The cause of the issue is the accidental deletion of all CRDs in the workload cluster, including addons without having an available etcd backup to restore them.

Resolution

Unfortunately, recovery isn’t possible in this case. Once a CRD is deleted, all associated CRs are removed from etcd. Without a backup, there is no supported path to restore them.

Additionally, the management (MGMT) cluster does not contain the workload cluster’s etcd data, CRDs, or CRs. This means we cannot force an add-on redeployment with the original data from the MGMT side.

Recommended next steps:

• If any etcd/cluster-state backups exist, restore from the most recent snapshot.
• If neither is available you will need to redeploy all clusters and applications.