Recurring DNS "Forwarder Upstream Server Timeout" alarm.

Products

• VMware NSX

Issue/Introduction

Title: Alarm for DNS Forwarder upstream server timeout
Event ID: dns.forwarder_upstream_server_timeout

Alarm Description

• Purpose: Indicates the DNS Forwarder querying timed-out to the upstream server.

• Impact: DNS requests to the DNS forwarder will time out.

Environment

VMware NSX

Cause

Typically occurs when the DNS Forwarder can't reach the upstream server or the server is unresponsive.

Resolution

Recommended Action: 

• Call nslookup API on the DNS forwarder to check the network issue or upstream server serving problem. This API request triggers a DNS lookup to the upstream server in the DNS forwarder's network namespace. <address> is the IP address or FQDN in the same domain as the upstream server. <enforcement-point-path> is the string path of enforcement point. If the API returns a connection timed out response, there is likely a network error or upstream server problem. Check why DNS lookups are not reaching the upstream server or why the upstream server is not returning a response.

  • GET /policy/api/v1/infra/tier-0s/{tier_0_id}/dns-forwarder/nslookup?address=<address>&enforcement_point_path=<enforcement-point-path>

  • GET /policy/api/v1/infra/tier-1s/{tier_1_id}/dns-forwarder/nslookup?address=<address>&enforcement_point_path=<enforcement-point-path>

• Run command from Edge node admin CLI: get dns-forwarder <dns-forwarder-uuid> live-debug server-ip <upstream-server-ip> to collect the querying's detailed information for debugging. Then collect the edge support bundle.

• Manually disable the DNS "Forwarder Upstream Server timeout" alarm on UI if it is disturbing.