PV fails to attach to a pod with error "CSINode ci-node-name does not contain driver csi.vsphere.vmware.com"

Products

• VMware Telco Cloud Automation

Issue/Introduction

• Unexpected shutdown of the backend storage, resulting in disconnection from the ESXi host.
• The kubectl describe command on the pod with the failed PV attachment shows below error: 
  

  kubectl describe pod pod-name -n namespace
  “Attach failed for volume "pvc-xxxxxx-xxxxx-xxxx-xxxxxxbd" : CSINode ci-node-name does not contain driver csi.vsphere.vmware.com”

Environment

TCA 2.3

Cause

Issue is caused by the missing “csi.vsphere.vmware.com” vsphere-csi driver in the worker node that happened due to corruption in configuration files after the backend storage outage.

Resolution

Restart the impacted Node where the driver is normally running. After that the PV will be accessible again and the other PODs will also successfully start.