How to estimate the number of log records emitted by TAS via rsyslog configured in Opsman UI
search cancel

How to estimate the number of log records emitted by TAS via rsyslog configured in Opsman UI

book

Article ID: 376961

calendar_today

Updated On:

Products

VMware Tanzu Application Service

Issue/Introduction

By default, all log records written to the /var/vcap/sys/log directories in all TAS VMs will be forwarded to the external syslog aggregation service if configured in Ops manager UI. This article provides a method to estimate the number of log records emitted by TAS which helps the capacity planning in the external rsyslog server.

Resolution

As the log file rotation is expected to happen once a threshold is reached, usually 50MB by default. We need to count the records in plain file as well as in compress file, and then sum them up.

Here is an example collecting the total number of records of a single job, router VM from 2024-09-07 20:00 to2024-09-07 23:59. The total log lines are 8841 + 5306 = 14147

  • plain text
bosh -d cf-03101f8#### ssh router/0 -c 'sudo grep "2024-09-07T2[0-9]" /var/vcap/sys/log -r|wc -l'|grep stdout
router/####: stdout | 8841
  • rotated file in gz format
bosh -d cf-03101f8#### ssh router/0 -c 'sudo find /var/vcap/sys/log -name \*.gz -print0 | sudo xargs -0 zgrep "2024-09-07T2[0-9]"|wc -l'|grep stdout
router/####: stdout | 5306

 

Regarding all the VMs under TAS tile, we can omit the VM name and it will iterate across all VMs automatically. Then we need to sum up all of them.

  • plain text
bosh -d cf-03101f8#### ssh  -c 'sudo grep "2024-09-07T2[0-9]" /var/vcap/sys/log -r|wc -l'|grep stdout
router/####: stdout | 8841
credhub/####: stdout | 643
nfs_server/####: stdout | 8636Connection to 10.225.61.67 closed.
nfs_server/####: stdout | 
log_cache/####: stdout | 1631
loggregator_trafficcontroller/####: stdout | 3195
mysql_monitor/####: stdout | 2734
cloud_controller_worker/####: stdout | 4286
doppler/####: stdout | 4947718
doppler/####: stdout | 
nats/####: stdout | 8579
mysql_proxy/####: stdout | 8157
diego_brain/####: stdout | 5035
uaa/####: stdout | 6509
clock_global/####: stdout | 1598
mysql/####: stdout | 7753
diego_cell/####: stdout | 9812
cloud_controller/####: stdout | 13785
diego_cell/####: stdout | 7017
  • rotated file in gz format
bosh -d cf-03101f8#### ssh  -c 'sudo find /var/vcap/sys/log -name \*.gz -print0 | sudo xargs -0 zgrep "2024-09-07T2[0-9]"|wc -l' |grep stdout
nfs_server/####: stdout | 0
cloud_controller/####: stdout | 0
loggregator_trafficcontroller/####: stdout | 0
doppler/####: stdout | 0
mysql_monitor/####: stdout | 0
credhub/####: stdout | 0
log_cache/####: stdout | 0
cloud_controller_worker/####: stdout | 0
nats/####: stdout | 0
mysql_proxy/####: stdout | 0
router/####: stdout | 5306
diego_brain/####: stdout | 0
mysql/####: stdout | 4818
clock_global/####: stdout | 1250
uaa/####: stdout | 15185
diego_cell/####: stdout | 4889
diego_cell/####: stdout | 3633
diego_database/####: stdout | 1997