OCP Cluster Fails to Authenticate to vCenter – CSI Driver Pods in CrashLoopBackOff

Products

• VMware vCenter Server

Issue/Introduction

^{In certain OpenShift Container Platform (OCP) clusters integrated with VMware vSphere, administrators may observe that the cluster fails to authenticate to the vCenter Server. As a result, the CSI driver pods enter a CrashLoopBackOff state, and containers restart approximately every hour before recovering automatically. This behavior can lead to degraded cluster performance, repeated error events, and limited ability to perform storage operations reliably.} 

^{vmware-vsphere-csi-driver-operator pod logs reveal repeated authentication errors:}

  error logging into vcenter: ServerFaultCode: Cannot complete login due to an incorrect user name or password.

  Failed to create govmomi client. err: error logging into vcenter: ServerFaultCode: Cannot complete login due to an incorrect user name or password.

  Marking vCenter connection status as false

  vsphere driver install failed with Failed to connect to vSphere: error logging into vcenter: ServerFaultCode: Cannot complete login due to an incorrect user name or password., found existing driver

  Marking cluster as degraded: vsphere_connection_failed error logging into vcenter: ServerFaultCode: Cannot complete login due to an incorrect user name or password.

  Marking cluster un-upgradeable because error logging into vcenter: ServerFaultCode: Cannot complete login due to an incorrect user name or password.

  VMwareVSphereController: error closing connection to vCenter API: no connection found to vcenter

 

 

Environment

^{CNS (Cloud Native Storage)}

^{VMware vCenter Server}

^{OpenShift Container Platform (OCP)} 

Cause

^{The CNS (Cloud Native Storage) user password mismatch, which prevents the CSI driver from successfully authenticating to vCenter. Without valid credentials, the driver cannot establish or maintain a stable connection, resulting in repeated pod restarts and degraded cluster state.}

Resolution

^{Two supported approaches can be used to resolve the issue:}

^{Option 1: Update vCenter Credentials and Refresh Services}

 

      ^{1. Update vCenter Password}

           ^{Update the vCenter password stored in the cluster configuration. Please engage the internal third‑party (OpenShift) team to perform the required steps.}

      ^{2. Restart the vmware-vapi-endpoint service on the vCenter Server:}

           ^{service-control --stop vmware-vapi-endpoint}

           ^{service-control --start vmware-vapi-endpoint}

      ^{3. Refresh Active Directory Integration}

           ^{Re‑join the ESXi host to the AD domain:}

           ^{/opt/likewise/bin/domainjoin-cli leave}

           ^{/opt/likewise/bin/domainjoin-cli join <domain_name> <username>}

 

^{Option 2: Create a Local User for CNS Role}

• ^{Create a local user in the vsphere.local domain.}
• ^{Assign the required privileges for the CNS role.}

Additional Information

^{For detailed guidance on CNS roles and privileges, refer to Broadcom documentation: 🔗 https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/vsphere-storage-8-0/getting-started-with-cloud-native-storage-in-vsphere/cloud-native-storage-for-vsphere-administrators/vsphere-cloud-native-storage-roles-and-privileges.html}