IPv6 ping packet may not be delivered on VMware NSX 4.1 logical router

Products

• VMware NSX Networking

Issue/Introduction

• IPv6 Pings from North to South (External to Internal) and South to North (Internal to External) may fail.
• When an internal device tries to reach and external, the logical router is used (VDR), this will use it virtual mac address, not the physical mac address in the Link Local Address Field of the packet.
• From NSX-T 4.1 onwards the edge has a command which allows you to flip this behavior so that the packet uses the source link layer address as the destination mac address.
• Sample packet, encapsulated inside Geneve packet arriving on the edge node:

<MAC> > <MAC>, ethertype IPv4 (0x0800), length 144: <IP>.51245 > <IP>.6081: Geneve, Flags [C], vni 0x11802, proto TEB (0x6558), options [8 bytes]: <MAC> > <MAC>, ethertype IPv6 (0x86dd), length 86: <IPv6>:4452 > <IPv6>:5300: ICMP6, neighbor solicitation, who has <IPv6>:5300, length 32
<base64>AAwpGGqwAFBWY4bJCABFBQCCAABAAEARVszIgClMyIApTcgtF8EAbgnXAkBlWAEYAgABBIABADgCADMz/1ZTAABQVuNFE4bdYFAAAAAgOv/+gAAAAAAAAABQVv/+VkRS/wIAAAAAAAAAAAAB/1ZTAIcASa8AAAAA/oAAAAAAAAAAUFb//lZTAAEBAlBWVkRS</base64>

• Sample NS, post encapsulation, arriving on the logical router:

<MAC> > <MAC>, ethertype IPv6 (0x86dd), length 86: <IPv6> > <IPv6> ICMP6, neighbor solicitation, who has <IPv6>, length 32
<base64>MzP/VlMAAFBW40UTht1gUAAAACA6//6AAAAAAAAAAFBW//5WRFL/AgAAAAAAAAAAAAH/VlMAhwBJrwAAAAD+gAAAAAAAAABQVv/+VlMAAQECUFZWRFI=</base64>

• Sample reply coming from the logical router inside the edge node, pre encapsulation:

<MAC> > <MAC>, ethertype IPv6 (0x86dd), length 86: <IPv6> > <IPv6>: ICMP6, neighbor advertisement, tgt is <IPv6>, length 32
<base64>AlBWVkRSAlBWVlMAht1gAAAAACA6//6AAAAAAAAAAFBW//5WUwD+gAAAAAAAAABQVv/+VkRSiAADNOAAAAD+gAAAAAAAAABQVv/+VlMAAgECUFZWUwA=</base64>

Environment

VMware NSX-T Data Center

Resolution

On each edge node, you can run the following command to flip this behavior.
Log in a root and run:

edge-appctl -t /var/run/vmware/edge/dpd.ctl lrouter/set_ns_lladdr_pmac enable

To revert this to default behavior again:

edge-appctl -t /var/run/vmware/edge/dpd.ctl lrouter/set_ns_lladdr_pmac disable

Note: This a system local command, therefore if an edge node is replaced, the commands will need to be entered again, that is if you have changed the default behavior to enabled now.

After upgrade to NSX-T 4.1, the edge can experience this issue, if so use the above command.