Creating new Cloud Native Storage (CNS) Persistent Volume Claims (PVC) fails. The issue correlates with placing an ESXi host connected to the shared datastore into Maintenance Mode.
The vsphere-csi-controller pod logs record the following errors:
rpc error: code = Internal desc = failed to create volume. Errors encountered: [ServerFaultCode: A specified parameter was not correct: InputSpec.datastores]
Error getting Persistent Volume for PVC ... with err: persistentvolume "" not found
VMware vCenter 8.0 Update 3
VMware ESXi 8.0 Update 3
Supervisor Cluster 1.32
The vSphere CSI driver relies on the datastore URL as the source of truth, while vCenter uses the Managed Object ID (MOID) to track accessibility. When a single datastore is mounted across multiple vSphere Datacenter objects, vCenter assigns a distinct MOID to the datastore within each Datacenter.
The CSI driver maintains an accessibility cache mapping the datastore URL to its MOIDs. If an ESXi host in a secondary Datacenter enters Maintenance Mode, the accessibility state for its specific MOID changes. Because multiple MOIDs share the same datastore URL, the state change inadvertently overwrites the driver's accessibility cache for the shared URL. This causes the driver to incorrectly evaluate the datastore as inaccessible across all clusters.
Workaround
Ensure CNS-backed datastores are not shared across multiple vSphere Datacenter objects.
Modify Fibre Channel (FC) zoning to restrict the datastore's visibility strictly to ESXi hosts within a single vSphere Datacenter.
If the environment is already in a failed state, removing the affected ESXi host from Maintenance Mode will temporarily restore PVC provisioning capabilities until the zoning changes can be implemented.
There has been a confirmed code defect regarding how the CSI driver processes datastore URLs and MOIDs for accessibility.
A permanent fix is under development; however, due to the significance of splitting the dsURL and MOID usage, no estimated timeframe for a resolution is currently available. The provided workaround must be utilized in the interim.