UMA metrics not appearing after deploying UMA v26.1.1.35 to EKS cluster.

Products

• DX SaaS

Issue/Introduction

Have an EKS cluster that we deployed UMA v26.1.1.35 to and after the deployment ,only a handful of metrics appeared in APM for the cluster. 

Waited over 3 hours but no new metrics appeared and then rolled back to UMA v25.4.2.5 which is working fine.

This same version is working fine in our OpenShift clusters. This is the first EKS cluster we are deploying to the 26.1.1.35 version

Environment

UMA version: 26.1.1.35

Kubernetes version: 1.34

Resolution

Seeing this error, in clusterinfo logs, which shows that it doesn't' have the sufficient permission to access each Kubernetes resources.

ERROR c.c.a.b.s.ClusterCrawlerService.log - error occurred in listNameSpaces. Err: , HTTP 403:  - {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces is forbidden: User \"system:serviceaccount:caapm:uma-clusterinfo-sa\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"namespaces"},"code":403}

ERROR c.c.a.b.s.ClusterCrawlerService.log - error occurred in listReplicasets. Err: , HTTP 403:  - {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"replicasets.apps is forbidden: User \"system:serviceaccount:caapm:uma-clusterinfo-sa\" cannot list resource \"replicasets\" in API group \"apps\" at the cluster

Though the deployment correctly defines the RBAC:

broadcom-uma-clusterinfo-clusterrole — grants get/list/watch on all required resources (pods, nodes, namespaces, services, events, configmaps, pvs, deployments, replicasets, daemonsets, statefulsets, cronjobs, resourcequotas)
broadcom-uma-clusterinfo-clusterrolebinding — binds that ClusterRole to uma-clusterinfo-sa in namespace caapm

# Check if ClusterRole exists
kubectl get clusterrole broadcom-uma-clusterinfo-clusterrole

# Check if ClusterRoleBinding exists
kubectl get clusterrolebinding broadcom-uma-clusterinfo-clusterrolebinding

# Verify the binding subject matches
kubectl describe clusterrolebinding broadcom-uma-clusterinfo-clusterrolebinding

# Test actual access from the SA perspective
kubectl auth can-i list pods \
  --as=system:serviceaccount:caapm:uma-clusterinfo-sa \
  --all-namespaces