How to identify apps on TAS which generate high log rate or consume high CPU with CF log-stream plugin

Products

• Operations Manager

Issue/Introduction

The article describes the steps to filter apps which are consuming high CPU resource or generating high rate of logs with CF CLI log-stream plugin. It is convenient for troubleshooting use if other monitor components are not installed.  

Environment

• TAS 2.0 and above
• Firehose v2 is enabled

Resolution

In order to use log-stream CF CLI plugin, please follow the instructions to install it. 

To list up apps/instances which are generating log rate > N bytes per second, for example 1000 as below. Please change the number as needed. 

$ cf log-stream -t gauge | egrep "app_id.*log_rate" | jq 'select(.gauge.metrics.log_rate.value > 1000) | {app_id: .tags.app_id, instance_id: .tags.instance_id, log_rate_bytes: .gauge.metrics.log_rate.value}'

{
  "app_id": "e2895927-c46a-44fa-80c0-17e679c15fbb",
  "instance_id": "0",
  "log_rate_bytes": 5380
  }
}
...

To list up apps/instances which are consuming CPU > percentage, for example 150% as below. Please change the number as needed. 

$ cf log-stream -t gauge | egrep "app_id.*cpu" | jq 'select(.gauge.metrics.cpu.value > 150) | {app_id: .tags.app_id, instance_id: .tags.instance_id, cpu_percentage: .gauge.metrics.cpu.value}'

{
  "app_id": "e2895927-c46a-44fa-80c0-17e679c15fbb",
  "instance_id": "0",
  "cpu_percentage": 348.8244049732760529
}
...