Velero restore TMC add-on
search cancel

Velero restore TMC add-on

book

Article ID: 313002

calendar_today

Updated On:

Products

Tanzu Kubernetes Grid

Issue/Introduction

What is best approach to backup and restore an add-on deployed from TMC using velero

Environment

VMware Tanzu Kubernetes Grid 1.x

Resolution

When backup of add-ons deployed with TMC with velero and then restore the pods and related components the restoration could restore the APP and package installs as we confirmed but TMC will not recognise the addon and eventually will fail.

 

Better approach would be to keep the configs from TMC as a backup and restore the config if needed. For harbor out to circumvent this issue is by performing below steps :

  • Instead of backing up the application and data we - only backup the PVC
  • When you need to restore, you restore the application to a cluster and use the existingClaim PVC option to reference the PVC that you have restored

In conclusion almost all Add-ons can be backed by storing the configs and for those having PVC use the method above. 

From the tests completed:
Deployed a harbor with TMc add-on and uploaded an image confirm the image is present and available for pull
Complete namespace backup of harbor namespace 
Delete the harbor app from TMC (make sure you safe the yaml config from TMC to prevent loss of FQDN and passwords used in the app) 
Once the app and the pods/pvc/pv are deleted  complete a restore to a new namespace only of the objects from that namespace

As a result there will be number of objects
PV   - need the PV objects as they contain the data the status of the PV is delete which will be removed after 
PVC
Pods
etc...
 

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                            STORAGECLASS         REASON   AGE 
persistentvolume/pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   5Gi        RWO            Delete           Bound    tanzu-system-registry-restored/pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   vwt-storage-policy            





Change the PV for each volume from Delate to Retain and remove the claim registration 
kubectl edit pv <name> 

End goal would look like this:

persistentvolume/pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   10Gi       RWO            Retain           Available                                                                    vwt-storage-policy            



Once changed you can delete the restored namespace to clear all objects 

Then verify if tanzu-system-registry namespace is created if not create it 
and for each PV from the above command create a PVC so the final output would look similar to below example for each pvc the name have to match the name of the pv and the claim size also have to match the storage allocated

Format of the PVC:

cat pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354
  namespace: tanzu-system-registry
spec:
  storageClassName: vwt-storage-policy
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi




kubectl get pv,pvc,pod -n tanzu-system-registry

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                            STORAGECLASS         REASON   AGE
persistentvolume/pvc-1855ceb0-ab60-4b67-9503-f25a2cecbc6e   5Gi        RWO            Retain           Bound    tanzu-system-registry/pvc-1855ceb0-ab60-4b67-9503-f25a2cecbc6e   vwt-storage-policy            68m
persistentvolume/pvc-38b5e858-5d71-44e8-8bf2-d89469ac6af2   1Gi        RWO            Retain           Bound    tanzu-system-registry/pvc-38b5e858-5d71-44e8-8bf2-d89469ac6af2   vwt-storage-policy            68m
persistentvolume/pvc-405716d3-8717-4f5e-a0e2-9fee7e5e54c7   1Gi        RWO            Retain           Bound    tanzu-system-registry/pvc-405716d3-8717-4f5e-a0e2-9fee7e5e54c7   vwt-storage-policy            68m
persistentvolume/pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   10Gi       RWO            Retain           Bound    tanzu-system-registry/pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   vwt-storage-policy            68m
persistentvolume/pvc-5dedc7da-68fa-422a-98fa-1c7793a2f1af   1Gi        RWO            Retain           Bound    tanzu-system-registry/pvc-5dedc7da-68fa-422a-98fa-1c7793a2f1af   vwt-storage-policy            68m

NAME                                                             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
persistentvolumeclaim/pvc-1855ceb0-ab60-4b67-9503-f25a2cecbc6e   Bound    pvc-1855ceb0-ab60-4b67-9503-f25a2cecbc6e   5Gi        RWO            vwt-storage-policy   3m24s
persistentvolumeclaim/pvc-38b5e858-5d71-44e8-8bf2-d89469ac6af2   Bound    pvc-38b5e858-5d71-44e8-8bf2-d89469ac6af2   1Gi        RWO            vwt-storage-policy   2m45s
persistentvolumeclaim/pvc-405716d3-8717-4f5e-a0e2-9fee7e5e54c7   Bound    pvc-405716d3-8717-4f5e-a0e2-9fee7e5e54c7   1Gi        RWO            vwt-storage-policy   2m21s
persistentvolumeclaim/pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   Bound    pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354   10Gi       RWO            vwt-storage-policy   77s
persistentvolumeclaim/pvc-5dedc7da-68fa-422a-98fa-1c7793a2f1af   Bound    pvc-5dedc7da-68fa-422a-98fa-1c7793a2f1af   1Gi        RWO            vwt-storage-policy   115s


From TMC

try and redeploy the add-on by adding the existing PVCs we have manually created here is example 

core:
  secret: VMware1!VMware1!
  xsrfKey: VMware1!VMware1!VMware1!VMware1!
database:
  password: VMware1!VMware1!
harborAdminPassword: VMware1!
hostname: VMware1.local
jobservice:
  secret: VMware1!VMware1!
persistence:
  imageChartStorage:
    s3:
      storageclass: vwt-storage-policy
  persistentVolumeClaim:
    database:
      existingClaim: pvc-405716d3-8717-4f5e-a0e2-9fee7e5e54c7
      storageClass: vwt-storage-policy
    jobservice:
      jobLog:
        existingClaim: pvc-1855ceb0-ab60-4b67-9503-f25a2cecbc6e
        storageClass: vwt-storage-policy
    redis:
      existingClaim: pvc-5dedc7da-68fa-422a-98fa-1c7793a2f1af
      storageClass: vwt-storage-policy
    registry:
      existingClaim: pvc-5b515782-cd03-4a18-b8f3-fd4a49f9d354
      storageClass: vwt-storage-policy
    trivy:
      existingClaim: pvc-1855ceb0-ab60-4b67-9503-f25a2cecbc6e
      storageClass: vwt-storage-policy
registry:
  secret: VMware1!
secretKey: VMware1!VMware1!


Where we have existing storage.

Verify if all pods will be started successfully and verify if the harbor page responds and the images are present.