In-band management configuration fails on Bare Metal Edge with Mellanox NICs

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:
The following conditions are met

- NSX-T Data Center 2.5.0 with Bare Metal Edge

- An attempt was made to configure in-band management on Mellanox NICs which failed with an error. Out of band management was configured instead.

- On the Edge Transport Nodes UI

    Configuration Status is "Success"
    Node Status is "Up"
    Tunnels are "Not Available"

- On the Edge cli the following commands show no output

    bmedge> get tunnel-ports
    bmedge> get vteps
    bmedge> get logical-router
    Logical Router
    UUID                  VRF    LR-ID  Name                    Type                Ports

- Edge syslog log messages are similar to this example

   2019-10-11T09:11:41 nsx-edge NSX 3631 FABRIC [comp="nsx-edge" level="ERROR" errorCode="EDG0400432"] kni alloc failed.
   2019-10-11T09:11:41 nsx-edge NSX 3631 FABRIC [comp="nsx-edge" level="ERROR" errorCode="EDG0400102"] Failed to create kni port eth0

Environment

VMware NSX-T Data Center 2.5.x
VMware NSX-T Data Center

Cause

In-band management is not supported on Mellanox NICs.
If an attempt is made to configure it, an error will be triggered however some stale configuration may block the remaining configuration of the Bare Metal Edge.

Resolution

This issue is resolved in NSX-T Data Center 2.5.1 and above.

Workaround:

Clear the stale data claiming the Melannox NIC for management

clear interface ethX.<MGMT_VLAN_ID>
restart service dataplane

Configure out-of-band management on the server or configure in-band management on a non-Mellanox NIC.