How to estimate the number of log records emitted by TAS via rsyslog configured in Opsman UI

Products

• VMware Tanzu Application Service

Issue/Introduction

By default, all log records written to the /var/vcap/sys/log directories in all TAS VMs will be forwarded to the external syslog aggregation service if configured in Ops manager UI. This article provides a method to estimate the number of log records emitted by TAS which helps the capacity planning in the external rsyslog server.

Resolution

As the log file rotation is expected to happen once a threshold is reached, usually 50MB by default. We need to count the records in plain file as well as in compress file, and then sum them up.

Here is an example collecting the total number of records of a single job, router VM from 2024-09-07 20:00 to2024-09-07 23:59. The total log lines are 8841 + 5306 = 14147

• plain text

bosh -d cf-03101f8#### ssh router/0 -c 'sudo grep "2024-09-07T2[0-9]" /var/vcap/sys/log -r|wc -l'|grep stdout router/####: stdout | 8841

• rotated file in gz format

bosh -d cf-03101f8#### ssh router/0 -c 'sudo find /var/vcap/sys/log -name \*.gz -print0 | sudo xargs -0 zgrep "2024-09-07T2[0-9]"|wc -l'|grep stdout router/####: stdout | 5306

 

Regarding all the VMs under TAS tile, we can omit the VM name and it will iterate across all VMs automatically. Then we need to sum up all of them.

• plain text

bosh -d cf-03101f8#### ssh  -c 'sudo grep "2024-09-07T2[0-9]" /var/vcap/sys/log -r|wc -l'|grep stdout router/####: stdout | 8841 credhub/####: stdout | 643 nfs_server/####: stdout | 8636Connection to 10.225.61.67 closed. nfs_server/####: stdout |  log_cache/####: stdout | 1631 loggregator_trafficcontroller/####: stdout | 3195 mysql_monitor/####: stdout | 2734 cloud_controller_worker/####: stdout | 4286 doppler/####: stdout | 4947718 doppler/####: stdout |  nats/####: stdout | 8579 mysql_proxy/####: stdout | 8157 diego_brain/####: stdout | 5035 uaa/####: stdout | 6509 clock_global/####: stdout | 1598 mysql/####: stdout | 7753 diego_cell/####: stdout | 9812 cloud_controller/####: stdout | 13785 diego_cell/####: stdout | 7017

• rotated file in gz format

bosh -d cf-03101f8#### ssh  -c 'sudo find /var/vcap/sys/log -name \*.gz -print0 | sudo xargs -0 zgrep "2024-09-07T2[0-9]"|wc -l' |grep stdout nfs_server/####: stdout | 0 cloud_controller/####: stdout | 0 loggregator_trafficcontroller/####: stdout | 0 doppler/####: stdout | 0 mysql_monitor/####: stdout | 0 credhub/####: stdout | 0 log_cache/####: stdout | 0 cloud_controller_worker/####: stdout | 0 nats/####: stdout | 0 mysql_proxy/####: stdout | 0 router/####: stdout | 5306 diego_brain/####: stdout | 0 mysql/####: stdout | 4818 clock_global/####: stdout | 1250 uaa/####: stdout | 15185 diego_cell/####: stdout | 4889 diego_cell/####: stdout | 3633 diego_database/####: stdout | 1997