How to confirm Geneve offload configuration on nenic driver

Products

• VMware vSphere ESXi
• VMware NSX

Issue/Introduction

• VMware NSX or VMware NSX-T Data Center has been deployed.
• Network cards are Cisco VIC NICs with support of Geneve offload.
• Geneve has been enabled and configured in UCS on VIC network adapter policy. 
• On the ESXi host You see that Geneve offload listed as enabled hardware capability on the NIC. Note vmnicX must match the actual vmnic, e.g. vmnic0 in example below:
  vsish -e get /net/pNics/vmnic0/properties | grep "Driver\|Device Hard\|Device Soft"
   Driver Name:nenic
   Driver Version:2.0.11.0
   Driver Firmware Version:5.3(3.85)
   Module Interface Used By The Driver:native
   Device Hardware Cap Supported: 0x483c032b -> VMNET_CAP_SG VMNET_CAP_IP4_CSUM VMNET_CAP_HIGH_DMA VMNET_CAP_TSO VMNET_CAP_HW_TX_VLAN VMNET_CAP_HW_RX_VLAN VMNET_CAP_SG_SPAN_PAGES VMNET_CAP_IP6_CSUM VMNET_CAP_TSO6 VMNET_CAP_TSO256k VMNET_CAP_GENEVE_OFFLOAD VMNET_CAP_SCHED
   Device Hardware Cap Supported (High): 0x61012c -> VMNET_CAP_PRIV_STATS VMNET_CAP_LINK_STATUS_SET VMNET_CAP_COALESCE_PARAMS VMNET_CAP_NETWORK_DUMP VMNET_CAP_OFFLOAD_ALIGN_ANY VMNET_CAP_MESSAGE_LEVEL VMNET_CAP_RING_PARAMS
   Device Hardware Cap Activated: 0x483c032b -> VMNET_CAP_SG VMNET_CAP_IP4_CSUM VMNET_CAP_HIGH_DMA VMNET_CAP_TSO VMNET_CAP_HW_TX_VLAN VMNET_CAP_HW_RX_VLAN VMNET_CAP_SG_SPAN_PAGES VMNET_CAP_IP6_CSUM VMNET_CAP_TSO6 VMNET_CAP_TSO256k VMNET_CAP_GENEVE_OFFLOAD VMNET_CAP_SCHED
   Device Software Cap Activated: 0xb7c00000 -> VMNET_CAP_UPT VMNET_CAP_RDONLY_INETHDRS VMNET_CAP_ENCAP VMNET_CAP_DCB VMNET_CAP_OFFSET_BASED_OFFLOAD VMNET_CAP_IP6_CSUM_EXT_HDRS VMNET_CAP_TSO6_EXT_HDRS VMNET_CAP_SRIOV
   Device Software Assistance Activated: 0 -> No matching defined enum value found.
   Device Hardware ENS Cap Supported: 0 -> No matching defined enum value found.
• In the ESXi host's vmkernel log, you can see logs indicating use of Geneve offload:
  /var/log/vmkernel.log
YYYY-MM-DDT09:19:37.848Z In(182) vmkernel: cpu5:2098092)nenic: enic_get_vnic_config:79: [0000:1b:00.3] vNIC csum tx/rx yes/yes, tso yes, vxlan no, geneve yes, rss yes, netqueue no, intr mode any, type min timer 125 usec, loopback tag 0x0000
YYYY-MM-DDT09:19:37.853Z In(182) vmkernel: cpu5:2098092)nenic: enic_geneve_offload_enable:1443: Geneve: Overlay offload enabled, udp_port = 6081

Environment

• VMware ESXi
• VMware NSX
• VMware NSX-T Data Center

Resolution

This is expected behaviour. 

Additional Information

In ESXi's vsi shell, you may notice that the rxqueue for vmnic backed by a VIC is not configured for Geneve filter, e.g.:

# vsish -e get /net/pNics/vmnic0/rxqueues/info | grep "filter classes"
   RX filter classes: 0x5 -> MAC VLAN_MAC

This feature is not related to Geneve offload:

• Geneve offload is used to offload the checksum calculation to the NIC. 
• Geneve filter (also known as NetQueue) is the rx feature used to filter packet flow with specific inner MAC address to one rx queue.
  This feature is not supported on nenic driver, only supported on nenic_ens driver. Ref. Cisco UCS Manager Network Management Guide, Release 4.1, section Geneve Offload. 

Additional resources: Cisco UCS Manager Network Management Guide Using the CLI, Release 4.1.