How to Calculate the Loggregator Message Throughput in Pivotal Cloud Foundry

Products

• VMware Tanzu Application Service for VMs

Environment

Resolution

Software Required

• cf cli
• Cloud Foundry Firehose plugin

The following commands should be run from the Ops Manager VM.

How to install the Cloud Foundry Firehose plugin

Installing the plugin is simple.

1. Log on to the CF CLI with your admin credentials:

cf api <your_cf_api_url>
cf login

2. Add the plugin repository:

cf add-plugin-repo CF-Community http://plugins.cloudfoundry.org/

3. Install the plugin:

cf install-plugin "Firehose Plugin" -r CF-Community

4. Install Pipe Viewer (PV):

apt-get install pv

How to Calculate Loggregator Message Throughput

1. Connect via SSH into the Ops Manager VM
2. Log on as the admin user with the cf cli
3. Execute `cf nozzle` and pipe to `pv` to calculate throughput. Please note the `egrep` statement is removing erroneous new line character from the `cf nozzle` command. In this example Loggregator is processing 460.4 messages a second. The output will refresh every 10 seconds. 

    cf nozzle -n | egrep -v "^$" | pv -l -i10 -r >/dev/null
   [460.4 /s]

 

Scaling Considerations

Refer to Loggregator Operators Guide for Scaling of Traffic Controllers, Dopplers, and Nozzles.  

On additional consideration is that Each Doppler can handle about 16k / envelopes per second max, so you should avoid exceeding 8-10k message per second from Metron. Metron is a process that forwards events, metrics, and logs to Dopplers and it exists on all PAS deployed instances. 

For example a Diego Cell emits app logs to Metron which in turn sends the to Doppler. You can measure the amount of envelopes emitted by a metron instance monitoring the metric "metron.egress". The Loggregator Operator's Guide provides more details about this metric.
 

VM Sizing

Refer to this Ops Guide for more Details on how these scaling decisions were made and what VM Types were used in testing. 

Additional Information

Pivotal Loggregator Ops Guide: https://docs.pivotal.io/pivotalcf/2-2/loggregator/log-ops-guide.html