Unable to provision new PVs in TKGi Cluster using CSI Storage class in a Topology Aware environment after upgrading to 1.18.3 with CSI Driver 3.1.x.

Products

• VMware Tanzu Kubernetes Grid Integrated (TKGi)
• VMware Tanzu Kubernetes Grid Integrated Edition
• VMware Tanzu Kubernetes Grid Integrated Edition (Core)
• VMware Tanzu Kubernetes Grid Integrated Edition 1.x
• VMware Tanzu Kubernetes Grid Integrated EditionStarter Pack (Core)

Issue/Introduction

When provisioning new PVs in a TKGi Cluster using CSI Storage class in a Topology Aware environment after upgrading from TKGi 1.17.4 to 1.18.3 and thus upgrading from CSI Driver 3.0.x to 3.1.x and below error is observed:

Error: failed to fetch hosts from entity Datastore:datastore-xx. Error: unrecognised entity type found Datastore:datastore-xx.

 

Environment

TKGi 1.18.3

CSI Driver 3.1.x

Cause

A Datastore associated to the TKGi Cluster had a label/tag that was blocking provisioning of new PVs using CSI Storage class in a Topology Aware environment.

In TKGi 1.18 which includes an embedded CSI Driver version 3.1.x, an enhancement for the validation of the topology tag exists.

If the tag is not attached to compute resource then validation will fail.

Resolution

1. Remove the associated Datastore tag via the tag options on the Datastore's summary page in the vCenter.
   
   
2. Restart the CSI Controller pod(s) for the Cluster.
   
   Or
   
   Recreate the affected TKGi Cluster using below command:
   
   

   bosh recreate -d service-instance_<cluster-id>

   
   Note: the above command will recreate each master and worker VM