unable to setup CARP or VRRP for virtual machine running on NSX segments

Products

• VMware NSX
• VMware vSphere ESXi

Issue/Introduction

• VMs connected to NSX segments (overlay or vLAN)
• CARP or VRRP MAC is not learnt on the segment MAC Table.
• Below vmnic drivers and firmwares are being used.

Mellanox Technologies ConnectX-5 EN NIC; OCP2.0, Type 1; 25GbE; dual-port SFP28; PCIe3.0 x8; (MCX542B-ACA)

Driver: nmlx5_core

Version: 4.23.6.5

Firmware Version: 16.30.1004

• Packet Capture taken on the associated ESXi vmnic of the vm showing the same VRRP packet on capture points UplinkSndKernel  and UplinkRcvkernel respectively.

 

16317 13:38:22.382782 #.#.#.# 224.0.0.18 VRRP 70 0x0000 (0) Announcement (v2)
Packet comments
CAPPT UplinkSndKernel HEADLEN 266 TSO 0 CSUM 0 CSUMVFD 0 ENCAP 0 ENCAPCSUMVFD 0 UDPRAND 0 OCSUM 0 SRCPORT ######## VLAN #### PRI 0 QID 0 SEGS 1 [ 70 ]
Frame 16317: Packet, 70 bytes on wire (560 bits), 70 bytes captured (560 bits) on interface unknown, id 0
Ethernet II, Src: IETF-VRRP-VRID (##:##:##:##:##:##), Dst: IPv4mcast (##:##:##:##:##:##)
Destination: IPv4mcast (##:##:##:##:##:##)
Source: IETF-VRRP-VRID (##:##:##:##:##:##)
Type: IPv4 (0x0800)
[Stream index: 79]
Internet Protocol Version 4, Src: #.#.#.#, Dst: 224.0.0.18
Virtual Router Redundancy Protocol

 

16318 13:38:22.382796 #.#.#.# 224.0.0.18 VRRP 70 0x0000 (0) Announcement (v2)
Packet comments
CAPPT UplinkRcvKernel HEADLEN 272 TSO 0 RSS TYPE 2 HASH ########## CSUM 0 CSUMVFD 0 ENCAP 0 ENCAPCSUMVFD 0 UDPRAND 0 OCSUM 0 VLAN #### PRI 0 SEGS 1 [ 9024 ]
Frame 16318: Packet, 70 bytes on wire (560 bits), 70 bytes captured (560 bits) on interface unknown, id 0
Ethernet II, Src: IETF-VRRP-VRID (##:##:##:##:##:##), Dst: IPv4mcast (##:##:##:##:##:##)
Destination: IPv4mcast (##:##:##:##:##:##)
Source: IETF-VRRP-VRID (##:##:##:##:##:##)
Type: IPv4 (0x0800)
[Stream index: 79]
Internet Protocol Version 4, Src: #.#.#.#, Dst: 224.0.0.18
Virtual Router Redundancy Protocol

Environment

VMware NSX

VMware vSphere ESXi

Cause

The packets are looped by the associated vmnic drivers

Resolution

Upgrade vmnic firmware version to latest version as per Broadcom Compatibility Matrix

 

Note: For drivers and firmwares versions of any associated vmnic, always make sure to install/update as per the Broadcom Compatibility Matrix depending on the current ESXi version.