False Positive Alarms frequently received as "Edge NIC Link Status Down" in NSX Manager web GUI

Products

• VMware NSX

Issue/Introduction

• Frequently receiving Alarm "Edge NIC Link Status Down" in UI even though the NIC is up
• Edge /var/log/syslog may report a permission error when accessing the operstate file
  <DATE>T<TIME>Z Edge NSX 3808 - [nsx@6876 comp="nsx-edge" subcomp="nsx-sha" username="nsx-sha" level="INFO" s2comp="fork-executor-2"] Exception catched when running cmd '{'cmd': ['sudo', 'cat', '/sys/class/net/<interface>/operstate'], 'input': None, 'shell': False, 'timeout': 4, 'env': None, 'type': 0, 'timestamp': 108137.569932216, 'seq': 29342}': {'seq': 29342, 'type': 0, 'executor': 2, 'timestamp': 108137.570239561, 'exception': PermissionError(1, 'Operation not permitted')}
• You may observe multiple eth0 nic down events in the syslog, even though the edge nodes are accessible via eth0 IP when you see the Alarm on the web GUI and get interface output confirms that the Link and admin status is UP
  
  syslog.3:2024-04-13T11:14:16.469Z <edge node> NSX 1184 - [nsx@6876 comp="nsx-edge" subcomp="nsx-sha" username="nsx-sha" level="CRITICAL" eventFeatureName="edge_health" eventType="edge_nic_link_status_down" eventSev="critical" eventState="On"] Edge node NIC eth0 link is down.

syslog.3:2024-04-13T11:34:37.677Z <edge node> NSX 1184 - [nsx@6876 comp="nsx-edge" subcomp="nsx-sha" username="nsx-sha" level="CRITICAL" eventFeatureName="edge_health" eventType="edge_nic_link_status_down" eventSev="critical" eventState="On"] Edge node NIC eth0 link is down.

syslog.3:2024-04-13T11:59:03.065Z <edge node> NSX 1184 - [nsx@6876 comp="nsx-edge" subcomp="nsx-sha" username="nsx-sha" level="CRITICAL" eventFeatureName="edge_health" eventType="edge_nic_link_status_down" eventSev="critical" eventState="On"] Edge node NIC eth0 link is down.
  
  
• From netstat -nape command output, you see that Edge eth0 ip is able to communicate with NSX Managers
  
  tcp        0      0 #.#.#.#:57688       #.#.#.#:1234        ESTABLISHED 155        17745      3162/nsx-proxy
tcp        0      0 #.#.#.#:35488       #.#.#.#:1235        ESTABLISHED 155        17747      3162/nsx-proxy
tcp        0      0 #.#.#.#:47496       #.#.#.#:1234        ESTABLISHED 155        17744      3162/nsx-proxy
tcp        0      0 #.#.#.#:46838       #.#.#.#:1234        ESTABLISHED 155        17743      3162/nsx-proxy

Environment

VMware NSX-T Data Center
VMware NSX

Cause

On the Edge accessing the file /sys/class/net/<interface>/operstate' occasionally returns a permission error. This causes the interface to be reported as down even though the interface is up and running. 

Resolution

There is no functional impact as these are false positives.

This issue is resolved in VMware NSX 3.2.4
This issue is resolved in VMware NSX 4.2.0