Disable VIP Authhub Prometheus monitoring during upgrade to version 3.1.

Products

• VIP Authentication Hub

Issue/Introduction

Sumologic observability is deployed in all EKS clusters and they are service monitors deployed with Prometheus.

This custom deployment will not use out of  box VIP Authhub Prometheus monitoring, instead using sumologic.

When out of box VIP Authhub Prometheus is not disabled, upgrade gets error:

hh:mm:ss  upgrade.go:131: [debug] performing update for ssp

hh:mm:ss  Error: UPGRADE FAILED: rendered manifests contain a resource that already exists. Unable to continue with update: could not get information about the resource: servicemonitors.monitoring.coreos.com "ssp-ssp-services-actuator-prometheus" is forbidden: User "example-user" cannot get resource "servicemonitors" in API group "monitoring.coreos.com" in the namespace "ssp"

hh:mm:ss  helm.go:81: [debug] servicemonitors.monitoring.coreos.com "ssp-ssp-services-actuator-prometheus" is forbidden: User "example-user" cannot get resource "servicemonitors" in API group "monitoring.coreos.com" in the namespace "ssp"

Tried to remove Prometheus portion configuration in values.yaml, attempt to upgrade to 3.1 again, received a different error:

hh:mm:ss  Error: UPGRADE FAILED: template: ssp/templates/prometheus-servicemonitor.yaml:20:20: executing "ssp/templates/prometheus-servicemonitor.yaml" at <.Values.ssp.global.monitoring.prometheusOperator.serviceMonitor.additionalLabels>: nil pointer evaluating interface {}.prometheusOperator
hh:mm:ss  helm.go:81: [debug] template: ssp/templates/prometheus-servicemonitor.yaml:20:20: executing "ssp/templates/prometheus-servicemonitor.yaml" at <.Values.ssp.global.monitoring.prometheusOperator.serviceMonitor.additionalLabels>: nil pointer evaluating interface {}.prometheusOperator
hh:mm:ss  UPGRADE FAILED

How can one disable VIP Authhub Prometheus monitoring during upgrade?

Environment

Operating System: AWS, EKS kubernetes 1.29
Vip auth hub version: 2.2.5 upgrade to 3.1 

Aurora PostgreSQL: 14.5

Cause

Custom deployment compatibility issue with 3rd party software.

Resolution

Try command "kubectl get servicemonitors -n ssp --as example-user", ensure account example-user does not have issue with clusterrolebinding permission.

In this customized deployment, the ssp chart is explicitly downloaded and modified. So when running the upgrade,  the following file "ssp/templates/prometheus-servicemonitor.yaml" should be removed before upgrade.