Alarm for LACP reporting member down

Products

VMware NSX

Issue/Introduction

Title: Alarm for LACP reporting member down
Event ID: transport_node_health.lag_member_down

Alarm Description:

Purpose: Monitoring lag member status on ESXi, KVM and Bare Metal Server; Please notice that KVM is unsupported from 4.0.0 release.
Impact: Lag member down will impact on connectivity and traffic flow of network.

Environment

VMware NSX-T Data Center

Resolution

Steps to resolve
For 3.0.0 and higher

Recommended Action:

Check the connection status of LAG members on hosts.

In the NSX UI navigate to "System | Fabric | Hosts | Clusters" or "System | Fabric | Hosts | Other Nodes" or "System | Fabric | Hosts | Standalone".
In the Nodes list of specific cluster or "Other Nodes" tab or "Standalone" tab, check the Status column. Find the Transport node with the degraded or down Node Status.
Select "<transport node>| View Details | Monitor". Check the status details of the PNIC/BOND which is reporting degraded or down.

Check the LACP member status details by logging into the failed host and invoking "esxcli network vswitch dvs vmware lacp status get" on an ESXi host or "ovs-appctl bond/show" and "ovs-appctl lacp/show" on a KVM / Linux Bare Metal Server host.

Esxi host command:
esxcli network vswitch dvs vmware lacp status get

Output example:

1-vds-1606
   DVSwitch: 1-vds-1606
   Flags: S - Device is sending Slow LACPDUs, F - Device is sending fast LACPDUs, A - Device is in active mode, P - Device is in passive mode
   LAGID: 2686314774
   Mode: Passive
   Nic List:
      Local Information:
      Admin Key: 9
      Flags: SP
      Oper Key: 9
      Port Number: 1
      Port Priority: 255
      Port State: AGG,DEF,
      Nic: vmnic1
      Partner Information:
      Age: 00:39:55
      Device ID: ##:##:##:##:##:00
      Flags: SP
      Oper Key: 0
      Port Number: 0
      Port Priority: 0
      Port State:
      State: Stand-alone

      Local Information:
      Admin Key: 9
      Flags: SP
      Oper Key: 9
      Port Number: 2
      Port Priority: 255
      Port State: AGG,DEF,
      Nic: vmnic2
      Partner Information:
      Age: 00:39:55
      Device ID: ##:##:##:##:##:00
      Flags: SP
      Oper Key: 0
      Port Number: 0
      Port Priority: 0
      Port State:
      State: Stand-alone

KVM / Linux bare metal server host command:
ovs-appctl bond/show

Output example:

---- nsx-uplink.0 ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 6
bond-hash-basis: 0
lb_output action: disabled, bond-id: -1
updelay: 0 ms
downdelay: 0 ms
next rebalance: 120 ms
lacp_status: configured
lacp_fallback_ab: false
active-backup primary: <none>
active member mac: ##:##:##:##:##:00(none)

member eth1: disabled
may_enable: false
member eth2: disabled
may_enable: false

KVM / Linux bare metal server host command:
ovs-appctl lacp/show

Output example:

---- nsx-uplink.0 ----
  status: active
  sys_id: ##:##:##:##:##:87
  sys_priority: 65534
  aggregation key: 3
  lacp_time: slow

member: eth1: defaulted detached
  port_id: 4
  port_priority: 65535
  may_enable: false

  actor sys_id: ##:##:##:##:##:87
  actor sys_priority: 65534
  actor port_id: 4
  actor port_priority: 65535
  actor key: 3
  actor state: activity aggregation collecting distributing defaulted

  partner sys_id: ##:##:##:##:##:00
  partner sys_priority: 0
  partner port_id: 0
  partner port_priority: 0
  partner key: 0
  partner state:

member: eth2: defaulted detached
  port_id: 3
  port_priority: 65535
  may_enable: false

  actor sys_id: ##:##:##:##:##:87
  actor sys_priority: 65534
  actor port_id: 3
  actor port_priority: 65535
  actor key: 3
  actor state: activity aggregation collecting distributing defaulted

  partner sys_id: ##:##:##:##:##:00
  partner sys_priority: 0
  partner port_id: 0
  partner port_priority: 0
  partner key: 0
  partner state: