"kubectl top node" shows "error: metrics not available yet" in a TKGi cluster

Products

• VMware Tanzu Kubernetes Grid Integrated (TKGi)

Issue/Introduction

"kubectl top node" shows "error: metrics not available yet", but "kubectl top pods" works fine.

Resolution

You can manually re-run the apply-specs errand for that particular Bosh deployment. If for some reason metrics-server wasn't successfully rolled out, this errand will roll it out again.

"deployment "metrics-server" successfully rolled out" would be expected in the output.

Example

Note "deployment "metrics-server" successfully rolled out" in the output below.

$ bosh -d service-instance_<cluster id> run-errand apply-specs
 
Instance   apply-addons/faa6d053-207a-4af7-9183-7d6b91ee854f
Exit Code  0
Stdout     No need to change the CoreDNS replica because there are 10 linux worker nodes
          Deploying /var/vcap/jobs/apply-specs/specs/coredns.yml
          serviceaccount/coredns unchanged
          clusterrole.rbac.authorization.k8s.io/system:coredns unchanged
          clusterrolebinding.rbac.authorization.k8s.io/system:coredns unchanged
          configmap/coredns unchanged
          deployment.apps/coredns unchanged
          service/kube-dns unchanged
          deployment "coredns" successfully rolled out
          Deploying /var/vcap/jobs/apply-specs/specs/metrics-server/
          clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator unchanged
          rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader unchanged
          apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io configured
          serviceaccount/metrics-server unchanged
          deployment.apps/metrics-server unchanged
          service/metrics-server unchanged
          clusterrole.rbac.authorization.k8s.io/system:metrics-server unchanged
          clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server unchanged
          secret/metrics-server-certs unchanged
          deployment "metrics-server" successfully rolled out
          System specs added successfully.
          Deploying /var/vcap/jobs/apply-specs/specs/addon-spec.yml
          namespace/pks-system unchanged
          deployment.apps/observability-manager unchanged
          clusterrole.rbac.authorization.k8s.io/observability-manager unchanged
          clusterrolebinding.rbac.authorization.k8s.io/observability-manager unchanged
          serviceaccount/observability-manager unchanged
          clusterrole.rbac.authorization.k8s.io/healthwatch unchanged
          clusterrolebinding.rbac.authorization.k8s.io/healthwatch configured
          all expected specs are running