Old persistent volumes still taking up space on datastore

Products

• VMware Integrated OpenStack

Issue/Introduction

• Seeing old persistent volumes taking up space in vCenter datastore browser
  /vmfs/volumes/########-####-####-####-############/kubevols/kubernetes-dynamic-pvc-########-####-####-####-############.vmdk
• Listing pv shows old backup persistent volumes
  #kubectl get pv -A
#kubectl get pvc -A

Environment

7.x

Resolution

• Since these are backup persistent volumes, check if the backup pod still exists and has completed running:
  #osctl get pods | grep backup*
  
  
• Check that the job(s) has completed running:
  #osctl get job | grep backup*
• If the pod doesn't exist and the job is completed, delete the persistentvolumeclaim
  #osctl delete pvc <pvc name>
• If the pod exists, delete the pod first and then the pvc as above.

Note: The persistent volume and the disk on vSphere should also delete automatically now.  Validate that this has occurred.

 

 

 

 

Additional Information

For manual backups, we may get see the PVC in Terminating status and stick there.  If we do see this, we can remove the finalizer to have it complete.

kubectl patch pvc {PVC_NAME} -p '{"metadata":{"finalizers":null}}'