Velero backup PartiallyFailed

Products

• Tanzu Kubernetes Runtime
• VMware vSphere Kubernetes Service

Issue/Introduction

• the velero backup get command shows error "PartiallyFailed"
• Velero backup fails when using the command velero backup create ######## --include-namespaces ######### 4h --snapshot-volumes
• Velero backup succeeds with the "--default-volumes-to-fs-backup" flag 
  
  logs show the below errors:
  kubectl api-resources:
  SnapshotCreationFailed                volumesnapshotcontent/snapcontent-<snapcontent id>                                                            Failed to create snapshot: failed to take snapshot of the volume file:<file id>: "rpc error: code = FailedPrecondition desc = queried volume doesn't have the expected volume type. Expected VolumeType: block. Queried VolumeType: FILE"
  
  Velero backup logs:
  could not sync content "snapcontent-<snapcontent id>": failed to take snapshot of the volume <volume id>: "rpc error: code = Internal desc = volumesnapshot:<snapcontent id> on namespace: <namespace> in supervisor cluster was not Ready. Error: volumesnapshot <volumesnapshot id> in namespace <namespace> not in ReadyToUse within 240 seconds: message: context deadline exceeded"
  

Environment

Velero

VMware vSphere Kubernetes Service

Cause

vSphere Container Storage Plug-in does not support Volume Snapshots for file volumes. It is only supported for block volumes:
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/container-storage-plugin/3-0/getting-started-with-vmware-vsphere-container-storage-plug-in-3-0/using-vsphere-container-storage-plug-in/volume-snapshot-and-restore.html

Resolution

when running "velero backup create" We have to specify flag "--default-volumes-to-fs-backup", then velero will use pod volume file system backup for volumes.

Run the following command:

velero backup create ##### --include-namespace ##### --default-volumes-to-fs-backup --include-cluster-resources=true --wait