LLDP information is not shown on Bare Metal Edge configured with Intel 810 XXV Nics with enhanced data path enabled

Products

• VMware NSX

Issue/Introduction

LLDP information is not shown for datpath interfaces (fp-eth0 to fp-eth3) on a Bare metal edge hardware with Intel 810 XXV Nics enabled with Enhanced data path 

However LLDP information is displayed for management interfaces (eth0 and eth1) configured with same Intel Nics.

Environment

4.2.2.1

Cause

On Bare Metal Edges with Intel NICs (X710/E810 series), the hardware has a built-in LLDP agent enabled by default to handle Data Center Bridging (DCB).

This built-in agent consumes the incoming LLDP packets. So the LLDP packet never reaches the DPDK driver in the fast path.

 

 

Resolution

Disable the hardware agent on using "ethtool". If the interface is currently active in DPDK mode, need to stop the Datapath service briefly to release the NIC back to the kernel. 

Steps are as below 

1. Log in as root in to BME using command 
   • st en
2. Stop the NSX Edge Datapath using command (This releases the NICs from DPDK (vfio-pci) back to the Linux Kernel driver (i40e / ice)).
   • systemctl stop nsx-edge 
3. Verify the interface is back in Kernel using command  (Normal standard interface names like eth0, eth1, etc., should be visible and be accessible))
   • ip link show
4. Check the Private Flags using command ( check if disable-fw-lldp shows off)
   • ethtool --show-priv-flags <ethX>
5. Disable the Firmware Agent. Run this for every fast-path interface (setting "disable" flag to "on" turns the agent OFF)
   • ethtool --set-priv-flags <ethX> disable-fw-lldp on
6. Start the NSX Edge Datapath:
   • systemctl start nsx-edge

The NICs will be grabbed back by DPDK, but the hardware setting to pass LLDP packets up will persist

Reboot the Edge node to Verify Persistence and check LLDP information again 

Note : On some Bare Metal Servers, a full reboot might reset the NIC registers to default (re-enabling the agent).

 

Additional Information

To verify the lldp information on BME login as admin and execute 

>> get physical-port <Interface-Name> lldp