Syslog configuration displays wrong FQDN on MySQL instances

Products

• VMware Tanzu Application Service
• VMware Tanzu for MySQL

Issue/Introduction

• In MySQL tiles deployed with syslog you see a strange FQDN in the output log messaging that doesn't appear to correspond with the bosh_dns provided FQDN.
• You are using the syslog_forwarder template to forward log messages from the application.
• The resulting /etc/rsyslog.d/25-syslog-release-forwarding-setup.conf file deployed on the MySQL VM will show the 'constant(value=' for the hostname as a structured query address instead of the bosh_dns provided FQDN:
  
  Example from 'bosh ssh' into MySQL VM:
  
  

  mysql/########-1234-45de-abcd-############ # cat /etc/rsyslog/25-rsyslog-release-forwarding-setup.conf
  
  
  set $.director = "";
  set $.deployment = "service-instance_########-4321-4d52-dcba-############";
  set $.instance = "mysql";
  set $.az = "az1";
  set $.id = "########-1234-45de-abcd-############";                ------------------> INSTANCE_ID
  # following https://tools.ietf.org/html/rfc5424#section-6
  template(name="SyslogForwarderTemplate" type="list") {
    constant(value="<")
    property(name="pri")
    constant(value=">1 ")
    property(name="timestamp" dateFormat="rfc3339")
    constant(value=" q-m163n1s0.q-g386.bosh ")
    property(name="app-name")
    constant(value=" ")
    property(name="procid")
    constant(value=" ")
    property(name="msgid")

  
  
  Expected 'constant(value=' address:
  
  ########-1234-45de-abcd-############.mysql.services.service-instance_########-4321-4d52-dcba-############.bosh

<INSTANCE___________ID____________>.<TILE>.<SERVICE>.<SERVICE______________INSTANCE________________ID>.bosh

 

 

Environment

MySQL tile installed in TAS for VMs. This is independent of versioning.

Cause

The MySQL tile utilizes bosh_dns differently than other tiles to present the currently active server when clustering is in use. This is an intentional architectural decision for the tile. The consequence of this architecture is that the syslog_forwarder utilizes the bosh_dns structured query address instead of the usual FQDN.

 

The resulting q-m163n1s0.q-g386.bosh name indicates:

 

m163= Numerical UUID 163

n1= Network 1

s0= Healthiness, specifically, the Default 'smart' query strategy

.q-g386= Group 386

 

This query presents the currently active instance in group 386, network 1, with numerical UUID 163, based on healthy and unchecked instances. The instance matching this query might vary if there are multiple MySQL VMs depending on server healthiness.

Resolution

Use the "id" field from resulting forwarded logs from the MySQL tile, in conjunction with the "deployment" to gather the VM and service-instance that match the source of the log message.

Use 'bosh -d <SERVICE-INSTANCE> vms' with the resulting deployment ID to find the MySQL VM.