TKGi csi-controller fails to start with error "Permission to perform this operation was denied"

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

The csi-controller is failing to start and as the other CSI components are dependant on it, they are also in a failed state.

The csi-controller.stderr.log log shows several errors while attempting to perform different operations on vCentre, but fails in all cases with "err: ServerFaultCode: Permission to perform this operation was denied."

Some examples are "failed to login to vc"

{"level":"error","time":"2025-11-23T12:48:46.930103264Z","caller":"vsphere/virtualcenter.go:196","msg":"failed to login to vc. err: ServerFaultCode: Permission to perform this operation was denied.","TraceId":"########-####-####-####-############","stacktrace":"sigs.k8s.io/vsphere-csi-driver/v3/pkg/common/cns-lib/vsphere.(*VirtualCenter)

And "Cannot connect to vCenter"

{"level":"error","time":"2025-11-23T12:48:46.930251679Z","caller":"vsphere/virtualcenter.go:270","msg":"Cannot connect to vCenter with err: ServerFaultCode: Permission to perform this operation was denied.","TraceId":"########-####-####-####-############","stacktrace":"sigs.k8s.io/vsphere-csi-driver/v3/pkg/common/cns-lib/vsphere.(*VirtualCenter).

And "failed to get vCenterInstance for vCenter"

{"level":"error","time":"2025-11-23T12:48:46.930318402Z","caller":"vanilla/controller.go:218","msg":"failed to get vCenterInstance for vCenter \"vcenter.example.com\"err=ServerFaultCode: Permission to perform this operation was denied.","TraceId":"########-####-####-####-##########",

 

Environment

TKGi with CSI

Cause

Using the credentials from /var/vcap/jobs/csi-controller/config/csi-vsphere.conf, an attempt to login to vSphere UI also fails.

This confirms that the vCentre user used by TKGI CSI doesn't have the necessary roles.

Resolution

Login to vSphere using an Admin user and assign the TKGi CSI user the Administrator role or the roles as outlined in "Creating Dedicated Users and Roles for vSphere"