Using same Kubernetes cluster-id in csi-vsphere.conf when using CSI driver, causing placement issues for container volumes
search cancel

Using same Kubernetes cluster-id in csi-vsphere.conf when using CSI driver, causing placement issues for container volumes

book

Article ID: 392054

calendar_today

Updated On: 03-26-2025

Products

VMware Cloud Native Runtimes

Issue/Introduction

When configuring the vSphere Container Storage Plug-in on a native Kubernetes cluster, when creating the csi-vsphere.conf if using multiple clusters, then cluster id should be unique. If not then this can cause issues with listing PVCs on datastores and with volumes creation on clusters

Example csi-vsphere.conf below

$ cat /etc/kubernetes/csi-vsphere.conf
[Global]
cluster-id = "<cluster-id>"
cluster-distribution = "<cluster-distribution>"
ca-file = <ca file path> # optional, use with insecure-flag set to false
thumbprint = "<cert thumbprint>" # optional, use with insecure-flag set to false without providing ca-file

[VirtualCenter "<IP or FQDN>"]
insecure-flag = "<true or false>"
user = "<username>"
password = "<password>"
port = "<port>"
datacenters = "<datacenter1-path>, <datacenter2-path>, ..."

 

 

Environment

vSphere 7.0

vSphere 8.0

Cause

Kubernetes cluster-id set in csi-vsphere.conf should be unique.

Resolution

From CSI driver 3.0, if you do not provide the cluster ID field or keep it empty while creating a configuration secret for vSphere Container Storage Plug-in, it automatically generates a unique cluster ID across all clusters.

See VMware vSphere Container Storage Plug-in 3

Steps to resolve are as follows. 

  1. Scale down the replicas of the vSphere CSI controller in all clusters using the duplicate cluster-id until only the valid cluster ID's CSI controllers remain running.
  2.  All of the CSI controllers in the Kubernetes clusters that had duplicate cluster IDs should be removed (not running)
  3. Assign a unique cluster-id in the vSphere configuration secret for each cluster where the duplicate ID was in use.
  4. For the CSI controller(s) still running on the cluster with the valid cluster ID -
  5. Wait for a full sync interval to complete (typically 30 minutes).
  6. During this time, the cluster will un-tag and de-register volumes associated with other Kubernetes clusters.
  7. In the vCenter CNS UI, confirm that the pvc volume counts from the cluster where valid cluster ID CSI controllers are running match expected pvc counts shown in the Kubernetes cluster.
  8. Once the volume counts are correct for the valid cluster ID environment-
  9. Scale up the replicas of the vSphere CSI controller in the other Kubernetes clusters where new, unique cluster-id values were configured.