How to increase the storage size for the Harbor Shared Service deployed in Tanzu Kubernetes Grid
Article ID: 317062
Updated On:
Products
Tanzu Kubernetes Grid
Issue/Introduction
The default storage size for Harbor in Tanzu Kubernetes Grid (TKG) is 10GB. This article provides instructions for increasing the storage size for the Harbor Shared Service deployed in Tanzu Kubernetes Grid.
Environment
VMware Tanzu Kubernetes Grid 2.x
Package deployed Harbor
Resolution
Please engage Broadcom support if assistance is required. The steps in the KB might lead to data loss if not performed correctly:
Note: Ensure that your kubectl context is set to the workload cluster where Harbor is deployed.
- Issue the following command to edit the harbor Extension and increase the synchronization interval to prevent automated reconciliation from overwriting any changes being made:
#kubectl -n tanzu-system-registry edit extension harbor - Find the line, syncPeriod: 5m, and change 5m (five minutes) to something much higher (at least 30m).
- Type :wq to exit the editor.
- Issue the following command to show the labels on the harbor-registry persistent volume claim (PVC):
# kubectl -n tanzu-system-registry get pvc --selector=component=registry --show-labels
Note: You will see output similar to the following. Make note of the values under the LABELS column:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE LABELS
harbor-registry Bound pvc-849bd401-aab5-4a01-9f54-78de1306f8d6 10Gi RWO default 13m app=harbor,component=registry,kapp.k14s.io/app=1610567506920108209,kapp.k14s.io/association=v1.034269eb21810ed9131cc41a27c729d4 - Create a file named harbor-registry-pvc.yaml with contents similar to the following:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
finalizers:
- kubernetes.io/pvc-protection
labels:
app: harbor
component: registry
kapp.k14s.io/app: "1610567506920108209"
kapp.k14s.io/association: v1.034269eb21810ed9131cc41a27c729d4
name: harbor-registry-100gb
namespace: tanzu-system-registry
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
storageClassName: default
volumeMode: Filesystem
Notes: The labels should match the LABELS values noted in Step 4. The name is arbitrary. Set the size (100GB in this example) appropriately for your needs. - Issue the following command to create the new PVC:
#kubectl -n tanzu-system-registry create -f harbor-registry-pvc.yaml - Issue the following command to remove the harbor-registry Pod:
#kubectl -n tanzu-system-registry scale deployment harbor-registry --replicas=0 - Issue the following command to edit the harbor-registry Deployment:
# kubectl -n tanzu-system-registry edit deployment harbor-registry - Find the volumeMounts section, which should look like the following:
volumeMounts:
- mountPath: /storage
name: registry-data
- mountPath: /etc/registry/passwd
name: registry-htpasswd
subPath: passwd
- mountPath: /etc/registry/config.yml
name: registry-config
subPath: config.yml
- mountPath: /etc/harbor/ssl/registry
name: registry-internal-certs - And add a new mountPath like the following:
- mountPath: /storage2
name: registry-data2
Repeat for the second volumeMounts section - Find the volume section, which should look like the following:
volumes:
- name: registry-htpasswd
secret:
defaultMode: 420
items:
- key: REGISTRY_HTPASSWD
path: passwd
secretName: harbor-registry-ver-1
- configMap:
defaultMode: 420
name: harbor-registry-ver-1
name: registry-config
- name: registry-data
persistentVolumeClaim:
claimName: harbor-registry
- name: registry-internal-certs
secret:
defaultMode: 420
secretName: harbor-registry-internal-tls
And add a new volume like the following:
- name: registry-data2
persistentVolumeClaim:
claimName: harbor-registry-100gb
Note: Make sure that the claimName value matches the name of the PVC created in Step 5. - Type :wq to exit the editor.
- Issue the following command to recreate the harbor-registry Pod:
#kubectl -n tanzu-system-registry scale deployments.apps harbor-registry --replicas=1 - Issue the following command to get the name of the harbor-registry Pod:
#kubectl -n tanzu-system-registry get po --selector=component=registry
Note: You will see output similar to the following:
NAME READY STATUS RESTARTS AGE
harbor-registry-86d87c9c66-qgz89 2/2 Running 0 7m41s - Issue a command similar to the following to exec into the harbor-registry Pod, using the Pod name obtained in Step 13:
# kubectl -n tanzu-system-registry exec -ti harbor-registry-86d87c9c66-qgz89 -- /bin/bash
Note: The prompt will change to harbor [ / ]$ if you have logged in successfully. - Issue the following command to copy the data from the original volume (/storage) to the new volume (/storage2):
# cp -rfp /storage/* /storage2/ - Type exit when the cp command is finished to exit the Pod.
- Issue the following command to delete the harbor-registry Pod:
#kubectl -n tanzu-system-registry scale deployment harbor-registry --replicas=0 - Issue the following command to edit the harbor-registry Deployment:
# kubectl -n tanzu-system-registry edit deployment harbor-registry - Remove the extra entries that were created in Step 10.
- Type :wq to exit the editor.
- Issue the following command to get the name of the harbor-registry persistent volume (PV):
# kubectl get pv |grep harbor-registry
Note: You will see output similar to the following:
pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 100Gi RWO Delete Bound tanzu-system-registry/harbor-registry-100gb
pvc-849bd401-aab5-4a01-9f54-78de1306f8d6 10Gi RWO Delete Bound tanzu-system-registry/harbor-registry
Note: For this example, the PV that we are interested in is the one that is 100GB in size, pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298. - Issue a command similar to the following to prevent the 100GB PV from being deleted if the associated PVC is deleted (replace pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 with the PV name noted in Step 20):
#kubectl patch pv pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' - Issue the following command to delete the harbor-registry PVCs:
#kubectl -n tanzu-system-registry delete pvc --selector=component=registry - Issue a command similar to the following to edit the harbor-registry PV, using the PV name noted in Step 20:
#kubectl edit pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 - Delete the entire claimref section, which should look similar to the following:
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
namespace: tanzu-system-registry/harbor-registry-100gb
resourceVersion: "28355"
uid: 0622e5ac-46c1-4a16-b908-b9b7445b4298 - Type :wq to exit the editor.
- Open the harbor-registry-pvc.yaml file with a text editor and change name to harbor-registry add a line similar to the following after volumeMode: Filesystem
volumeName: pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298
Note: Replace pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 with the PV name noted in Step 20. - Issue the following command to recreate the harbor-registry PVC:
#kubectl apply -f harbor-registry-100gb.yaml - Issue the following command to recreate the harbor-registry Pod:
#kubectl -n tanzu-system-registry scale deployment harbor-registry --replicas=1 - Issue a command similar to the following to allow the harbor-registry PV to be deleted when the harbor-registry PVC is deleted (replace pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 with the PV name noted in Step 20):
#kubectl patch pv pvc-0622e5ac-46c1-4a16-b908-b9b7445b4298 -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}' - Issue the following command to edit the harbor Extension and set the synchronization interval back to the default:
#kubectl -n tanzu-system-registry edit extension harbor - Change the value for syncPeriod: back to 5m (five minutes).
- Type :wq to exit the editor.
- Issue the following command to get the name of the harbor-registry Pod:
#kubectl -n tanzu-system-registry get po --selector=component=registry
Note: You will see output similar to the following:
NAME READY STATUS RESTARTS AGE
harbor-registry-64ddb7b98c-4dhl9 2/2 Running 0 7m41s - Issue a command similar to the following to exec into the harbor-registry Pod, using the Pod name obtained in Step 33:
# kubectl -n tanzu-system-registry exec -ti harbor-registry-64ddb7b98c-4dhl9 -- /bin/bash
Note: The prompt will change to harbor [ / ]$ if you have logged in successfully. - Issue the following command to validate that the /storage volume is reporting the proper capacity:
# df -h /storage
Note: You will see output similar to the following:
Filesystem Size Used Avail Use% Mounted on
/dev/sde 98G 61M 93G 1% /storage - Log in to the Harbor UI to validate that the proper images are present.
Additional Information
Impact/Risks:
The data that exists in Harbor can be lost if these steps are not followed explicitly.
Note: The Extensions method is deprecated. Harbor and other platform components are now deployed and managed as packages/apps using the tanzu package CLI. To modify configurations such as storage size, you must create an overlay Secret and update the corresponding PackageInstall resource.
The data that exists in Harbor can be lost if these steps are not followed explicitly.
Note: The Extensions method is deprecated. Harbor and other platform components are now deployed and managed as packages/apps using the tanzu package CLI. To modify configurations such as storage size, you must create an overlay Secret and update the corresponding PackageInstall resource.
Feedback
Yes
No
Powered by
