Effects of enabling Automatic Health Check in an environment via NSX

Products

VMware NSX VMware vSphere ESXi

Issue/Introduction

The NSX API and interface allow the enabling of environmental health checks. This function leverages 0x8922-type probing packets to determine the connectivity status between NSX side and physical side.

These health check packets validate the MTU size and VLAN accessibility within the environment.

Environment

VMware NSX

VMware vSphere ESXi

Cause

This is a feature that is disabled by default. Enabling this feature requires toggling "Health Configuration" found in "System → Fabric → Transport Zones → Health Configuration" in the NSX graphic interface. This feature can also be toggled on/off by using the API described in Management Plane API > Troubleshooting And Monitoring > Healthcheck. When enabled, this feature functions globally, on all hosts assigned to NSX Transport Zones.

Feature enabled:

Feature disabled (default):

Resolution

What do these health checks do?

This feature is cross ported from vSphere. The basic function of this feature is to detect VLAN & MTU mismatch configuration between NSX side and physical side.
The description of this feature can be found in the product documentation at Health Check VLAN ID Ranges and MTU Settings. and Enabling vSphere Distributed Switch health check in the vSphere Web Client.

This feature has two modes: Manual and Automatic.

Manual mode is for performing the healthcheck once. This mode is purely available via API call. Documentation on how to invoke the healthcheck can be found in the API guide at Management Plane API > Troubleshooting And Monitoring > Healthcheck
Automatic mode is for performing the healthcheck periodically. For now, the interval of the automatic mode is 15 minutes. Users can leverage the toggle under "Transport Zone->Automatic Health Check->Turn On" to enable the automatic mode, and can only check the result via the API at System Administration > Monitoring > Health Checks.

To verify within an environment that this feature is enabled, perform the following checks:

On an NSX prepared ESXi host, review the /var/log/nsx-syslog.* files for "Received VLAN-MTU check request for DVS" messages. This log line will be recorded every 15 minutes, and the associated packets will be sent at the same cadence.

#grep "Received VLAN-MTU check request for DVS" /var/log/nsx-syslog.log
2024-07-26T22:25:59.446Z nsx-opsagent[1052536]: NSX 1052536 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="healthcheck" tid="1053286" level="INFO"] Received VLAN-MTU check request for DVS <VDS-NAME>
2024-07-26T22:40:59.745Z nsx-opsagent[1052536]: NSX 1052536 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="healthcheck" tid="1053288" level="INFO"] Received VLAN-MTU check request for DVS <VDS-NAME>
Via API (Automatic mode only, toggled on/off via GUI)
- To see the toggle status:
  GET https://<nsxmanagerfqdn>/api/v1/automatic-health-check-toggle
- To see the full list of locations / transport zones where health check has been enabled:
  GET https://<nsxmanagerfqdn>/api/v1/automatic-health-checks/
- To see the result of a single transport zone's health check data:
  GET https://<nsxmanagerfqdn>api/v1/automatic-health-checks/transport-zones/<transport-zone-id>
These packets can be captured at an ESXi host, entering or leaving, by using the following commands:
- For inbound traffic: pktcap-uw --uplink <vmnic-name> --capture UplinkRcvKernel --ethtype 0x8922 -P -o </path/to/storage/rx_pkts_file_name>.pcap
- For outbound traffic: pktcap-uw --uplink <vmnic-name> --capture UplinkSndKernel --ethtype 0x8922 -P -o </path/to/storage/tx_pkts_file_name>.pcap
In a network packet capture the following packets will be visible:
- Note: these are layer 2 packets and have no IP address. The Source/Destinations are the Shadow/Virtual MAC address of the vmnics throughout the environment.
To determine the shadow/virtual MAC of an ESXi host (the highlighted line shows the same MAC address as is in the capture screenshot above):
- On a host without NSX installed:
  # esxcfg-info -n | grep -B 6 "Virtual MAC"
  |----Name..................................................vmnic0
  |----PCI Segment...........................................0
  |----PCI Bus...............................................11
  |----PCI Slot..............................................0
  |----PCI function..........................................0
  |----MAC Address...........................................##:##:##:##:##:##
  |----Virtual MAC Address...................................00:50:56:59:cb:6d
  
  |----Name..................................................vmnic1
  |----PCI Segment...........................................0
  |----PCI Bus...............................................19
  |----PCI Slot..............................................0
  |----PCI function..........................................0
  |----MAC Address...........................................##:##:##:##:##:##
  |----Virtual MAC Address...................................00:50:56:58:a9:0c
- On a host with NSX installed:
  
  # nsxdp-cli vswitch instance list
  DvsPortset-0 (<VDS-NAME>) ## ## ## ## ## ## ## ##-## ## ## ## ## ## ## ##
  
  Total Ports:3460 Available:3447
  
    Client PortID DVPortID MAC Uplink VID VNI
  
    Management ######## ##:##:##:##:##:## n/a N/A N/A
  
    vmnic0 ########## ### ##:##:##:##:##:## 0-4094 N/A
  
    Shadow of vmnic0 67108870 00:50:56:59:cb:6d n/a N/A N/A
  
    vmnic1 ########## ### ##:##:##:##:##:## 0-4094 N/A
  
    Shadow of vmnic1 67108872 00:50:56:58:a9:0c n/a N/A N/A
  
    vmk0 ######## ## ##:##:##:##:##:## vmnic0 115 0
  
    vmk1 ######## ## ##:##:##:##:##:## vmnic1 112 0
  
    vmk10 ######## ########-####-####-####-############ ##:##:##:##:##:## vmnic0 141 0
  
    vmk11 ########   ########-####-####-####-############ ##:##:##:##:##:## vmnic1 141 0
  
    vmk2 ######## ## ##:##:##:##:##:## vmnic0 111 0
  
    vmk50 ########   ########-####-####-####-############ ##:##:##:##:##:## void 0
  
    vdr-vdrPort ######## vdrPort ##:##:##:##:##:## vmnic1 0