PVC fails to provision persistent volumes on vSAN Cluster with error - "Waiting for a volume to be created either by the external provisioner 'csi.vsphere.vmware.com' or manually by the system administrator"
search cancel

PVC fails to provision persistent volumes on vSAN Cluster with error - "Waiting for a volume to be created either by the external provisioner 'csi.vsphere.vmware.com' or manually by the system administrator"

book

Article ID: 427013

calendar_today

Updated On:

Products

VMware vSAN Tanzu Kubernetes Runtime

Issue/Introduction

  • PVC fails to provision persistent volumes for OpenShift on vSAN Cluster
  • Error Message:

Warning  ProvisioningFailed 64s csi.vsphere.vmware.com_vmware-vsphere-csi-driver-controller-745cb688df-nk4gk_51dedfa3-e493-481f-####-############  failed to provision volume with StorageClass "test": rpc error: code = Internal desc = failed to create volume. Error: failed to create volume with fault: "(*types.LocalizedMethodFault)(0xc00172c300)({\n DynamicData: (types.DynamicData) {\n },\n Fault: (types.CnsFault) {\n  BaseMethodFault: (types.BaseMethodFault) <nil>,\n  Reason: (string) (len=16) \"VSLM task failed\"\n },\n LocalizedMessage: (string) (len=32) \"CnsFault error: VSLM task failed\"\n})\n"   Normal   ExternalProvisioning  2m41s (x167 over 42m)  persistentvolume-controller Waiting for a volume to be created either by the external provisioner 'csi.vsphere.vmware.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. Normal Provisioning 9s (x17 over 42m) csi.vsphere.vmware.com_vmware-vsphere-csi-driver-controller-745cb688df-nk4gk_51dedfa3-e493-481f-####-############  External provisioner is provisioning volume for claim "openshift-monitoring/alertmanager-pvc-alertmanager-main-1"

  • StorageClass has been created and points to the correct backend
  • CSI driver components are deployed
  • Required secrets and credentials have been configured
  • Worker nodes show the correct topology/storage labels

 

Environment

  • VMware vSAN OSA
  • Kubernetes Cluster

Cause

  • The issue is caused by inconsistent space efficiency settings between the vSAN Cluster and the Storage Policy used for the PVC
  • This inconsistency causes object creation failure on vSAN

 

  • /var/log/vpxd/vpxd.log for the failure:

2026-01-23T11:02:38.415Z error vpxd[06802] [Originator@6876 sub=Default opID=9b236def-1] [VpxLRO] -- ERROR task-3729924 -- 527ebba8-253f-71f5-236e-684936becc79(525bb149-47f2-2562-7a4a-5461eec13473) -- VStorageObjectManager -- vim.vslm.vcenter.VStorageObjectManager.createDisk: :vmodl.fault.SystemError
--> Result:
--> (vmodl.fault.SystemError) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    reason = "Failed to create namespace: nameSpacePath = /vmfs/volumes/vsan:52958714d8aab4a7-################/catalog
--> Failed to init fcd-catalog for datastore: datastorePath->Value() = /vmfs/volumes/vsan:52958714d8aab4a7-################/"
-->    msg = "A general system error occurred: Failed to create namespace: nameSpacePath = /vmfs/volumes/vsan:52958714d8aab4a7-##############/catalog
--> Failed to init fcd-catalog for datastore: datastorePath->Value() = /vmfs/volumes/vsan:52958714d8aab4a7-################/"
--> }
--> Args:
-->
--> Arg spec:
--> (vim.vslm.CreateSpec) {
-->    name = "pvc-d722f6d9-b488-4f61-####-############",
-->    keepAfterDeleteVm = true,
-->    backingSpec = (vim.vslm.CreateSpec.DiskFileBackingSpec) {
-->       datastore = 'vim.Datastore:vsan:52958714d8aab4a7-################',

 

  • /var/log/vsanvcmgmtd.log on the ESXi host for the failure:

2026-01-23T11:01:56.127Z info vsanvcmgmtd[2088302] [vSAN@6876 sub=CnsVolMgr opID=9b236def] Creating volume with spec: (vim.cns.VolumeCreateSpec) [
2026-01-23T11:01:56.158Z info vsanvcmgmtd[2107416] [vSAN@6876 sub=Workflow opID=9b236def] Creating volume task started
2026-01-23T11:02:38.426Z error vsanvcmgmtd[2088275] [vSAN@6876 sub=Workflow opId=9b236def] Workflow current action has fault (vim.fault.CnsFault) {
-->    faultCause = (vmodl.fault.SystemError) {
-->       faultCause = (vmodl.MethodFault) null,
-->       faultMessage = <unset>,
-->       reason = "Failed to create namespace: nameSpacePath = /vmfs/volumes/vsan:52958714d8aab4a7-################/catalog
--> Failed to init fcd-catalog for datastore: datastorePath->Value() = /vmfs/volumes/vsan:52958714d8aab4a7-################/"
-->       msg = "A general system error occurred: Failed to create namespace: nameSpacePath = /vmfs/volumes/vsan:52958714d8aab4a7-################/catalog
--> Failed to init fcd-catalog for datastore: datastorePath->Value() = /vmfs/volumes/vsan:52958714d8aab4a7-################/"
-->    },
-->    faultMessage = <unset>,
-->    reason = "VSLM task failed"
-->    msg = ""
--> }

  • /var/log/hostd.log on the ESXi host for the failure:

2026-01-23T11:02:36.398Z In(166) Hostd[2105053] [Originator@6876 sub=Libs opID=9b236def-1-63-63ff sid=5236931c user=vpxuser:VCLOUD.LOCAL\vpxd-extension-fa5040cc-ccbc-4ab2-b28d-10dc2c877cd6] OSFSIpcCall: END (opIDS = ) IPC returned error : Failure
2026-01-23T11:02:36.398Z In(166) Hostd[2105053] [Originator@6876 sub=Libs opID=9b236def-1-63-63ff sid=5236931c user=vpxuser:VCLOUD.LOCAL\vpxd-extension-fa5040cc-ccbc-4ab2-b28d-10dc2c877cd6] Vob Stack:
2026-01-23T11:02:36.398Z In(166) Hostd[2105053] [Originator@6876 sub=Libs opID=9b236def-1-63-63ff sid=5236931c user=vpxuser:VCLOUD.LOCAL\vpxd-extension-fa5040cc-ccbc-4ab2-b28d-10dc2c877cd6] [vob.vsan.clomd.incompatibleDatastoreSpaceEfficiency]: Object policy is not compatible with datastore space efficiency policy configured on the cluster
2026-01-23T11:02:36.398Z In(166) Hostd[2105053] [Originator@6876 sub=Libs opID=9b236def-1-63-63ff sid=5236931c user=vpxuser:VCLOUD.LOCAL\vpxd-extension-fa5040cc-ccbc-4ab2-b28d-10dc2c877cd6] [vob.vsanprovider.object.creation.failed]: Failed to create object.

  • Policy on the device group has deduplication and compression disabled:

   Device: naa.5002############
   Display Name: naa.5002############
   Is SSD: true
   VSAN UUID: 5206c354-bd28-1400-####-############
   VSAN Disk Group UUID: 5207e2c0-3ed3-6769-####-############
   VSAN Disk Group Name: naa.5000############
   Used by this host: true
   In CMMDS: true
   On-disk format version: 20
   Deduplication: false
   Compression: false
   Checksum: 15980785581228981423
   Checksum OK: true
   Is Capacity Tier: true
   Encryption Metadata Checksum OK: true
   Encryption: false
   DiskKeyLoaded: false
   Is Mounted: true
   Creation Time: Fri Oct  3 12:46:07 2025

Resolution

  • For vSAN OSA, Storage Policies cannot enforce settings for Space Efficiency on the vSAN Cluster
  • Space Efficiency settings on the vSAN cluster for vSAN OSA is always the setting configured on the cluster
  • Change the Space Efficiency rule in the Storage Policy to "No Preference"
  • This will ensure that the policy uses the Space Efficiency setting of the cluster and will ensure compatibility of the space efficiency settings between the storage policy and the cluster

Note: Do not use any other option for Space Efficiency in the Storage Policy for vSAN OSA 

Powered by Wolken Software