PCF Metrics 1.6.4 is in a redirect loop when trying to log in
search cancel

PCF Metrics 1.6.4 is in a redirect loop when trying to log in

book

Article ID: 293381

calendar_today

Updated On:

Products

Operations Manager

Issue/Introduction

When user tries to log into the Metrics UI with valid login information, the user is redirected back to the login page. It doesn't matter how many times you log in the user is continuously forced back to the login page.

This article provides a solution for a bug recently discovered in the Metrics 1.6.4 app where the SKIP_CERT_VERIFY option is not working but also describes various other reasons why you could encounter this redirect loop. You can skip to the resolution section if you know for certain you are hitting this bug.


 

Cause

The reason for this Issue can be a few things, such as the UAA authentication and authorization flow not authorizing you with the proper scopes:

During UAA authentication and authorization the user is checked to see whether or not they have the permission to the scopes that are being requested. The metrics client apps_metrics  is usually granted the following scopes:

  • cloud_controller.read 
  • cloud_controller.admin

Usually this is auto-approved for the metrics client, but if for some reason it's not and these scopes have been marked as DENY for the metrics client in the UAA database, it's possible you will end up in a redirect loop. 

mysql> select * from authz_approvals;
+------------+--------------+------------------------+---------------------+----------+---------------------+------------------+
| user_id    | client_id    | scope                  | expiresat           | status   | lastmodifiedat      | identity_zone_id |
+------------+--------------+------------------------+---------------------+----------+---------------------+------------------+
| 4a2e478d   | apps_metrics | cloud_controller.admin | 2020-07-08 02:52:40 | DENIED   | 2020-06-08 02:52:40 | uaa              |
| 6a7e070d   | apps_metrics | cloud_controller.read  | 2020-07-08 02:52:40 | DENIED   | 2020-06-08 02:52:40 | uaa              |
+------------+--------------+------------------------+---------------------+----------+---------------------+------------------+

 

There are many other reasons why the UAA authentication and authorization flow would deny you access which ultimately ends up in a redirect loop for the Metrics UI, Failing the CSRF check or even a SAML error can result in a redirect loop.

But the most likely cause you end up in a re-direct loop would be due to a certificate error. The following are some of the areas you should check to see where the problem may lie.

  1. Subject alternative names (SANs) may not be defined properly for the frontend LB or gorouter.  Check that it follows the following scheme: 
    *.yourdomain.com,*.apps.yourdomain.com,*.system.yourdomain.com,*.login.system.yourdomain.com,*.uaa.system.yourdomain.com
  2. Check that you have added any required Root CA's have been define in the Bosh director under the security section and that you have checked to include the OpsMan root certificate on all the vm's:
  3. Check that none of the certs being used for the load-balancer (LB) or gorouter have expired. You can use the following openssl command to check. Replace <hostname-or-ip> with the component that is terminating TLS. Usually this will be your load-balancer or gorouter. 
    openssl s_client -showcerts -connect <hostname-or-ip>:443 < /dev/null | openssl x509 -noout -text


Resolution

Resolution

Upgrade to the latest version of App Metrics 2.0.x


Workaround

If there is a valid reason to skip ssl verification for metrics you have 2 possibilities. Both of them involve setting an environment variable on the metrics app found in the System org and the metrics-v1-6 Space.

Getting apps in org system / space metrics-v1-6 as admin...
OK

name               requested state   instances   memory   disk   
metrics-queue      started           1/1         512M     1G     
logs-queue         started           1/1         256M     1G
metrics-ingestor   started           1/1         512M     1G     
metrics            started           1/1         1G       2G     
metrics-ui         started           1/1         256M     1G     
metrics-alerting   started           1/1         1G       2G
monitors-api       started           1/1         1G       2G

 

  1. Set the environment variable SKIP_SSL_VALIDATION to true for the metrics app. What this essentially does is to set all calls made to UAA to be HTTP only. This may or may not be a valid solution depending on whether or not your load balancer allows un-encrypted traffic on port 80, plus there are the obvious security pitfalls when using this method.   
    cf set-env metrics SKIP_SSL_VALIDATION true
  2. The second option is to set SKIP_CERT_VERIFY to true, this will skip certificate verification but will still encrypt all traffic. Unfortunately this flag does not seem to be working in Metrics 1.6.4. As an alternative you can set the following environment variable on the metrics app in order to skip cert verification. 
    cf set-env metrics NODE_TLS_REJECT_UNAUTHORIZED 0
    Note: This option is less secure and should only be used if you have a valid reasons to skip certificate verification in Metrics 1.6.4.


Powered by Wolken Software