Velero Scheduled Backup Fails with Validation and Rate Limiter Timeout Errors

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

Scheduled Velero backups for Persistent Volumes (PVs) and Persistent Volume Claims (PVCs) fail. Initial backup attempts return a rate limiter timeout: 

`<error getting backup volume info: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline>`

Environment

vSphere Kubernetes Service

Cause

The Velero Schedule Custom Resource is misconfigured. Common causes include:
1. Deploying the Schedule resource in a non-standard application namespace rather than the `velero` namespace.
2. Explicitly referencing a non-existent backup storage location (e.g., "default") in the YAML, which overrides the global default flag.
3. Manually specifying a Volume Snapshot Location that does not exist or is not required for the vSphere CSI plugin.

Resolution

1. Recreate the Velero Schedule manifest to deploy in the `velero` namespace.
2. Update the storageLocation parameter in the Schedule YAML to match the exact name of the configured BackupStorageLocation, or remove the parameter entirely to allow Velero to use the backup storage location marked as default.
3. Remove the volumeSnapshotLocations block from the Schedule YAML. The vSphere CSI plugin will automatically utilize the configured Kubernetes VolumeSnapshotClass.
4. Apply the corrected Schedule:
kubectl apply -f <schedule_manifest>.yaml