SNMP timeouts and UDP packet loss on overlay segments

Products

• VMware NSX

Issue/Introduction

Intermittent network connectivity issues and timeouts may occur when application pods (e.g., Zabbix) attempt to poll external devices via SNMP (UDP port 161) from an NSX environment. This article addresses troubleshooting steps to identify if the root cause is external physical network security filtering.

Environment

VMware NSX

SNMP-based monitoring applications (e.g., Zabbix)

Cause

Physical network security appliances may incorrectly identify bursts of legitimate SNMP UDP traffic as a "UDP flood" event, causing these devices to actively drop SNMP response packets before they reach the ESXi physical uplinks. Because SNMP uses stateless UDP, there is no retransmission, resulting in application timeouts.

Resolution

1. Perform a packet capture (pcap) at the worker node level (eth0) to verify if SNMP requests are leaving the source pods.
2. Compare the request count with the corresponding response count. If requests are sent but responses are intermittently not returning, packet loss is occurring upstream.
3. Coordinate with the physical network security or firewall team to investigate their network security appliance logs.
4. Verify if the security appliance is flagging legitimate SNMP response traffic as a "UDP flood" or security threat.
5. If identified, request that the security team adjust the UDP flood protection thresholds on the physical network to accommodate the volume of SNMP polling traffic.

For further details on how to perform a packet capture within the NSX environment use the following article: Troubleshooting Packet Captures for VMware NSX

Additional Information

If the issue persists after physical network verification, for further assistance, use our main Contact Broadcom Support page.