Understanding Path MTU Discovery (PMTUD) in VMware SD-WAN by VeloCloud

Products

• VMware SD-WAN by VeloCloud

Issue/Introduction

Path MTU Discovery (PMTUD) can consist of different algorithms and protocols. The purpose of this document is to explain the functionality within the context of VMware SD-WAN Edges.

Environment

VMware SD-WAN by VeloCloud

Resolution

Prior to the Edge version 3.3.x PMTUD was carried out entirely via RFC 1191 (https://tools.ietf.org/html/rfc1191). Essentially what this protocol does is send traffic with the Don't Fragment (DF) bit set and relies on receiving ICMP "Datagram too big" messages to determine when an MTU is too large, and what the correct MTU is. In some cases (e.g. USB modems), this response is not received and the MTU must be manually configured.

Beginning in the Edge release 3.3.x PMTUD process has been enhanced. If RFC 1191 fails, then it falls back to a separate algorithm to ensure manual setting of the MTU is not required, RFC 4821 (https://tools.ietf.org/html/rfc4821). In this mode, the edge does a binary search sending packets of varying sizes and self-acknowledging them to discover the MTU.

PMTUD is run per-path every time a Overlay path comes UP, either for the first time, or after it has gone dead (i.e. flapped).

For Public WAN links: The MTU chosen will be the lowest MTU of all the paths to various VMware SD-WAN Gateways
For Private WAN links: The MTU chosen will be the lowest MTU of all paths to all the gateways (if they exist) and to other edges.

It's also important to note that the lowest MTU of all the WAN links is used as the standard MTU for that edge, as it's needed to have a standard MTU size for packets to be able to load balance effectively across all WAN links.

MTU can be manually configured by choosing the Overlay in the WAN Link Configuration section, and clicking on the Advanced tab:



Regarding setting the MTU manually:

• This does not turn off PMTUD. Whenever a path comes up after a flap, PMTUD will still run, and if it detects a lower MTU than what is configured, it will use the lower MTU.
• If you set the MTU to a value lower than the previous one, then it will simply use the lower MTU.
• However, if you set MTU to a higher value than the previous one, it will tear down the paths and rebuild, which will also trigger PMTUD to run again. So it's not possible to manually set the MTU to be higher than the MTU that PMTUD discovers.

The MTU may also be configured under the physical interface by going to Configure > Device > Interface Settings 

If the physical interface has a low MTU set, that will logically affect any tunnels built through that interface. So when troubleshooting you should check the MTU on both the physical interface, and the overlay in WAN Settings.

Additional Information

To learn more about how we handle packets that are too large check out this KB article: