How to verify metrics are reaching the Autoscaler
search cancel

How to verify metrics are reaching the Autoscaler

book

Article ID: 297577

calendar_today

Updated On:

Products

VMware Tanzu Application Service for VMs

Issue/Introduction

Identify the cause why the Autoscaler is not scaling when expected.

The first thing you to determine is whether the relevant metrics (timer events originating from Gorouter) are reaching LogCache, where the Autoscaler requests them. In order to simulate the Autoscaler requests, you'll use the LogCache plugin for the cf CLI.

 


Environment


Cause

Metrics may not be passed to Autoscaler if LogCache is undersized, metrics are being lost, or if it is having other problems.

 

Resolution

Follow these steps to validate Autoscaler functionality:

  1. SSH to the affected system.
  2. Install the log-cache-cli cf cli plugin located at: `https://github.com/cloudfoundry/log-cache-cli`
  3. Run cf log-meta. Confirm that the output indicates a Cache Duration of longer than 30 minutes for the application trying to be scaled. If the output is less than 30 minutes, check the Count field and see if it's hit 100,000. You will need to raise the max_per_source value, which is exposed in PAS 2.4.7+ , 2.5.3+ and 2.6.0+.
  4. Capture the log envelopes from LogCache directly. Run cf tail -t timer -f --json <customer-app> > timer-envelopes.json.
  5. Generate some load on the application.
  6. Review the envelopes.json file. This should contain one timer entry for each request that you submitted.


Additional Information

When metrics are not reaching Autoscaler, then it is recommended to review the loggregator system which passes metrics as log messages. Oftentimes missing metrics are indicative of the problem in the loggregator system. See the following articles:



Powered by Wolken Software