BFD tunnels are showing down because of "min_rx_ttl configuration set incorrectly

Products

• VMware NSX

Issue/Introduction

• Multi-hop BFD sessions are down due to a small TTL.
• Edge BFD sessions are down, suggesting no Rx packets.
• Ping works between the tunnels.
• From the packet captures on the edge, it is noticed that the Multihop BFD packets are received on the TEP interface

• The rx drops count increases against the Rx_drop_rx_ttl_small
edge01> get bfd-session local-ip 192.z.y.x remote-ip 192.x.y.z stats
BFD Session
Rx_drop                       : 0
Rx_drop_rx_ttl_small          : 0

Environment

VMware NSX

Cause

A custom Edge Cluster Profiles configuration is applied to the NSX Edges with a lower 'BFD Allowed Hops'. 

The lower 'BFD Allowed Hops' sets the ttl to a lower value and if the TEPs are multiple hops away, the packet will be dropped.

Resolution

To fix this, the min_rx_ttl configuration on the edge needs to be adjusted to accept packets, depending on the network path and expected TTL decrement.

For Example :- 

From support bundle of NSX Edge you see that min_rx_ttl is 252.
    {
       "local_address": "10.#.#.#",
       "remote_address": "10.#.#.#",
        "type": "IPv4",
       "interface": "d91####bf-####-5635-b000-###333##333",
        "session_type": "VLAN",
        "intf_type": "TUNNEL",
        "encap": "vlan",
        "local_discr": "2098248190",
        "remote_discr": "0",
        "min_rx_ttl": 252, <<<<<<<<<<<<<<<<


So the edge is expecting the TTL should be minimum of 252 but you are getting packets with TTL 251. So edge is dropping the packet.

Follow below steps to correct the configuration :-

Log in to the NSX Manager UI and follow the steps below:

1. Navigate to System.

2. Go to Edge Cluster Profiles.

3. Check the value for BFD Allowed Hops.

4. Edit the setting and update BFD Allowed Hops .

5. After making the change, please verify the status.

6. Tunnels should be up now.