NSX-T Geneve BFD Tunnels Down Due to Physical Router MTU Misconfiguration

Products

• VMware NSX

Issue/Introduction

VMware NSX-T Data Center (NSX-T) virtual machine traffic reaches the Distributed Logical Router (DLR) but is blackholed and fails to egress the host. Additionally, Edge-to-Host and Edge-to-Edge Bidirectional Forwarding Detection (BFD) tunnels remain in a DOWN state. Active/Standby Edge node failovers may result in total traffic loss for the Virtual Network Identifier (VNI). Application-level symptoms may include Operations Nodes or Analytics services failing to come online due to the underlying overlay network unavailability.

Environment

• VMware NSX-T Data Center 9.0.2

Cause

The physical transit router for the Tunnel Endpoint (TEP) VLAN is configured with an MTU below the minimum required threshold to support Geneve encapsulation overhead, causing the physical underlay to silently drop encapsulated BFD keepalive packets.

Resolution

• Access the management interface of the physical router or switch routing the TEP VLAN.

• Review the interface configuration settings for the paths connecting the ESXi host TEPs and Edge Node TEPs.

• Reconfigure the interface Maximum Transmission Unit (MTU) to a minimum of 1600 bytes end-to-end (1700 bytes or 9000 bytes for Jumbo Frames is highly recommended to accommodate Geneve overhead).

• Save the configuration on the physical network devices.

• Verify within the NSX Manager UI (or via the NSX CLI) that the BFD tunnel status for Edge-to-Host and Edge-to-Edge connections has transitioned to an UP state and overlay datapath traffic is restored.