vRealize Log Insight generates a large number of DNS queries to DNS servers
search cancel

vRealize Log Insight generates a large number of DNS queries to DNS servers

book

Article ID: 315938

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

Symptoms:
  • There is a high number of DNS requests from vRealize Log Insight to the internal DNS server(s), upwards of 50% of all queries


Environment

VMware vRealize Log Insight 8.1.x

Resolution

This is a known issue affecting vRealize Log Insight, there is no resolution at this time.  Please subscribe to this article to be informed when updates are published.

Workaround:
To workaround this issue, follow one of the beneath options for the ingestion source(s) of your choosing

Note: It is recommended to disable/adjust these features gradually, rather than all in one sitting.  For example, if you've gone through disabling/adjusting the syslog value and still feel displeased with the outcome, then proceed with adjusting the API server and vRealize Operations integration.

Syslog Server

How to adjust:

  1. Navigate to https://vRealizeLogInsightIPorFQDN/internal/config and under the following section, increase the dns-cache-entries value for syslog

Note: Replace vRealizeLogInsightIPorFQDN with the IP or Fully Qualified Domain Name (FQDN) of your environment.

<importer>
.
.
.
<dns-cache-entries value="102400" />
</importer>

Note: The value for dns-cache-entries parameter should be set to higher than the expected spikes in the number of hosts logging to Log Insight. There is currently no auto adjustment mechanism for this parameter.

  1. SSH to the primary node as root
Note: If you do not know your root password, refer to: How to reset the root password in vRealize Log Insight (53649)
  1. Restart the service by running the following command
service loginsight restart
  1. Repeat steps 2-3 sequentially on all nodes in the vRealize Log Insight cluster.

How to disable:

  1. Navigate to https://vRealizeLogInsightIPorFQDN/internal/config and under the following section, set the syslog-resolve-source value value to false.

Note: Replace vRealizeLogInsightIPorFQDN with the IP or Fully Qualified Domain Name (FQDN) of your environment.

<import>
.
.
.
<syslog-resolve-source value="false" />
</import>

  1. SSH to the primary node as root

Note: If you do not know your root password, refer to: How to reset the root password in vRealize Log Insight (53649)

  1. Restart the service by running the following command

service loginsight restart

  1. Repeat steps 2-3 sequentially on all nodes in the vRealize Log Insight cluster.

API Server

How to adjust:

  1. Navigate to https://vRealizeLogInsightIPorFQDN/internal/config and under the following section, increase the dns-cache-entries value for API:.

Note: Replace vRealizeLogInsightIPorFQDN with the IP or Fully Qualified Domain Name (FQDN) of your environment.

 <api-server>
.
.
.
<dns-cache-entries value="102400" />
</api-server>

Note: The value for dns-cache-entries parameter should be set to higher than the expected spikes in the number of hosts logging to Log Insight. There is currently no auto adjustment mechanism for this parameter.

  1. SSH to the primary node as root
Note: If you do not know your root password, refer to: How to reset the root password in vRealize Log Insight (53649)
  1. Restart the service by running the following command
service loginsight restart
  1. Repeat steps 2-3 sequentially on all nodes in the vRealize Log Insight cluster.


How to disable:

  1. Navigate to https://vRealizeLogInsightIPorFQDN/internal/config and under the following section, set the api-resolve-source value to false.
Note: Replace vRealizeLogInsightIPorFQDN with the IP or Fully Qualified Domain Name (FQDN) of your environment.
.....
<api-server>
.
.
.
<api-resolve-source value="false" />
</api-server>
  1. SSH to the primary node as root
Note: If you do not know your root password, refer to: How to reset the root password in vRealize Log Insight (53649)
  1. Restart the service by running the following command
service loginsight restart
  1. Repeat steps 2-3 sequentially on all nodes in the vRealize Log Insight cluster.

vRealize Operations

How to adjust:

In the case where vRealize Operations integration is configured in vRealize Log Insight, the frequency of inventory updates can be decreased with the following steps:

  1. Navigate to https://vRealizeLogInsightIPorFQDN/internal/config
  2. Under the following section, increase the poll-seconds value to 3600 or higher
<vcops>
<enabled value="true" />
....
<poll-seconds value="900" />
....
</vcops>
 
Notes: This value determines the frequency of sending DNS requests to build the inventory mapping between objects in vRealize Operations, and logs ingested by vRealize Log Insight.
 
This will not always be effective, and we are exploring other options for mitigating the issue when it occurs due to integration with vRealize Operations
  1. SSH to the primary node as root
Note: If you do not know your root password, refer to: How to reset the root password in vRealize Log Insight (53649)
  1. Restart the service by running the following command
service loginsight restart
  1. Repeat steps 3-4 sequentially on all nodes in the vRealize Log Insight cluster.

How to disable:

  1. Navigate to https://vRealizeLogInsightIPorFQDN/admin
  2. Under Integration, click on vRealize Operations
  3. Click the x in the top right corner to remove the integration

How to adjust in vRealize Log Insight 8.8.2:

As of vRealize Log Insight 8.8.2, there is a new property in the <vcops> section of internal/config. It's called "matching-policy". It's used to control DNS queries done while trying to match vRealize Operations resource with the source log event.

There are 3 possible values:
  • legacy
  • smart
  • off
The default value is "smart". With this option, matching will be done based on resource type. The number of queries would be 2-3 times smaller than with the "legacy" option, but the accuracy might be reduced.
"Legacy" is the same method which was in used until vRealize Log Insight 8.8.2. It does a lot of DNS queries and has the highest accuracy.
"Off" completely disables DNS queries, but not inventory matching. This has the lowest accuracy.

Which option to choose highly depends on the environment. These options offer a tradeoff between accuracy and number of DNS queries. As such, the value should be chosen based on priorities. If DNS queries are flooding your network, switch it to "off". If high number of DNS queries are fine, but the accuracy of matching is lacking, switch it to "legacy".

To change this setting, follow the steps below:
  1. Navigate to https://vRealizeLogInsightIPorFQDN/internal/config
  2. Locate the section under the <vcops> tag
  3. Modify the value for matching-policy as you see fit
  4. Click Save


Additional Information

Impact/Risks:
These workarounds will affect how vRealize Log Insight resolves host names, including hostname of the syslog messages, Launch in Context, and Alert Integration features of the vRealize Operations integration.

Powered by Wolken Software