PVC creation fails with error "VSLM task failed" returned by CSI controller

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

When creating or resizing a PVC(especially very large one) on TKGI, it fails and below error message is recorded in vsphere-csi-controller logs

{"level":"error","time":"2025-01-13T07:18:43.134335345Z","caller":"volume/util.go:297","msg":"failed to create volume with fault: "(*types.LocalizedMethodFault)(0xc0010f8ee0)({
 DynamicData: (types.DynamicData) {
 },
 Fault: (types.CnsFault) {
  BaseMethodFault: (types.BaseMethodFault) <nil>,
  Reason: (string) (len=16) "VSLM task failed"
 },
 LocalizedMessage: (string) (len=32) "CnsFault error: VSLM task failed"
 })
...

 

Environment

TKGI all versions

Cause

VSLM (vSphere Storage Lifecycle Manager) is responsible for creating, cloning, and deleting FCDs, which are used by CNS volumes. When you create a PVC) in Kubernetes, the vSphere CSI driver interacts with CNS, and CNS uses VSLM to create the underlying disk.

Here are some common reasons for "VSLM task failed"

• Insufficient storage capacity in datastores
• The vSphere user used by the CSI driver lacks necessary permissions for CNS/VSLM operations.
• vCenter Task Timeout or Lock

Resolution

Please locate the fail task on vCenter at "Monitor ->Tasks and Events" and check the failure details. Based on the detailed reason, resolve the storage capacity, permission or timeout issue accordingly.