Error: Internal error occurred: # default StorageClasses were found (reason: Forbidden)

Products

• VMware Telco Cloud Automation

Issue/Introduction

• We get this error message when trying to deploy the Prometheus add-on:
  

  PackageInstallReady:Prometheus package install is not ready. I1103 <Time>   ##### request.go:###] Waited for <Time-in-Seconds>s due to client-side throttling, 
  not priority and fairness, request: GET:https://192.168.1.1:443/apis/monitoring.telco.vmware.com/v1alpha1 kapp: Error: create persistentvolumeclaim/alertmanager (v1) namespace: tanzu-system-monitoring: Creating resource persistentvolumeclaim/alertmanager (v1) namespace: tanzu-system-monitoring: API server says: persistentvolumeclaims "alertmanager" is forbidden: Internal error occurred: 2 default StorageClasses were found (reason: Forbidden):

Environment

3.2

Cause

The primary cause is the existence of multiple default Kubernetes StorageClasses, which violates the requirement for single, unambiguous default dynamic provisioning in a Kubernetes cluster, preventing the PersistentVolumeClaim (PVC) for Alertmanager from binding.

The Kubernetes API server returns a Forbidden error because it cannot automatically provision a volume when two StorageClasses are marked as default.

Resolution

The solution is to unset the "Default flag" on one of the StorageClasses using the Telco Cloud Automation (TCA) UI

1. List all StorageClasses and identify the default ones. Look for the StorageClass where the annotation storageclass.kubernetes.io/is-default-class is set to true.
   

   kubectl get sc

2. Navigate to the TCA UI.

3. Go to the cluster in question and then the vsphere-csi-addon configuration section.

4. Locate the StorageClass settings.

5. Change the Is Default flag to False for the StorageClass