Prometheus addon deployment on a specific node pool

Products

• VMware Telco Cloud Automation

Issue/Introduction

Steps to deploy Prometheus add-on in CLI

Environment

3.1,3.2

Resolution

Steps to deploy Prometheus on a specific node pool:

1. SSH to workload cluster
2. Create prom.yaml with the following:

#@ load("@ytt:overlay", "overlay")

#@ deployment_names = ["prometheus-server", "alertmanager", "prometheus-pushgateway"]

#@ for name in deployment_names:
#@overlay/match by=overlay.subset({"kind": "Deployment", "metadata": {"name": name}})
---
spec:
  template:
    spec:
      #@overlay/match missing_ok=True
      nodeSelector:
        telco.vmware.com/nodepool: #####-###-###-###-prometheus
#@ end
#@overlay/match by=overlay.subset({"kind": "Deployment", "metadata": {"name": "prometheus-kube-state-metrics"}})
---
spec:
  template:
    spec:
      #@overlay/match missing_ok=True
      #@overlay/replace
      nodeSelector:
        telco.vmware.com/nodepool: #####-###-###-###-prometheus

Note: replace #####-###-###-###-prometheus with the actual node pool name

3. Execute below command post .yaml file creation

kubectl create secret generic prom -n tca-system -o yaml --dry-run=client --from-file=prom.yaml |kubectl apply -f -

kubectl annotate pkgi -n tca-system   prometheus ext.packaging.carvel.dev/ytt-paths-from-secret-name.0=prom

4. Run the following command to check if node Selector is set

kubectl get deploy -n tanzu-system-monitoring   prometheus-server -o jsonpath='{.spec.template.spec.nodeSelector}'

5. If nodeSelector is not set after 15 minutes, check prometheus app to see if there is any error:

kubectl get app -n tca-system   prometheus -oyaml