NSX ESXi transport node Errors and Application Crash Due to OOM (Out of Memory)
search cancel

NSX ESXi transport node Errors and Application Crash Due to OOM (Out of Memory)

book

Article ID: 428365

calendar_today

Updated On:

Products

VMware NSX VMware vSphere ESXi

Issue/Introduction

  • This article addresses issues related to multiple host errors and application crashes, specifically the nsx-exporter service.
  • These errors occur due to Out of Memory (OOM) conditions leading to service failures. The logs show memory exhaustion and related issues that result in core dumps and system instability.
  • You will see nsx-exporter core-dump file generated in /var/core

  • Similar log entries can be seen in the following files:

    /var/run/log/vmkernel.log showing OOM for nsx-exporter:

    vmkernel: cpu0:2105489)Admission failure in path: host/vim/vmvisor/nsx-exporter:python.40556761:sched.40556761
    vmkernel: cpu0:2105489)sched.40556761 (322964606) requires 44 KB, asked 44 KB from nsx-exporter (50518) which has 786432 KB occupied and 0 KB available.
    vmkernel: cpu0:2105489)Admission failure in path: host/vim/vmvisor/nsx-exporter:python.40556761:sched.40556761
    vmkernel: cpu0:2105489)sched.40556761 (322964606) requires 44 KB, asked 44 KB from nsx-exporter (50518) which has 786432 KB occupied and 0 KB available.
    vmkernel: cpu13:2105488)Admission failure in path: host/vim/vmvisor/nsx-exporter:nsx-exporter.2105425:uw.2105425
    2026-01-23T12:00:50.912Z In(182) vmkernel: cpu13:2105488)started from 'sh' 2105415 with cmdline '/bin/sh /sbin/watchdog.sh -s NSX_EXPORTER -q 20 /usr/lib64/vmware/nsx-exporter/nsx-exporter ++securitydom=49', parent 2097773

    /var/run/log/vmsyslogd-dropped.log

    17632026-01-18T08:41:25.425Znsx-exporter2105425NSX 2105425 - [nsx@6876 comp="nsx-esx" subcomp="agg-service" tid="2105488" level="ERROR" errorCode="MPA11006"] Unable to collect flows for filter nic-2134837-eth1-vmware-sfw.2, rc=8
    17632026-01-18T08:41:25.428Znsx-exporter2105425NSX 2105425 - [nsx@6876 comp="nsx-esx" subcomp="agg-service" tid="2105488" level="ERROR" errorCode="MPA11006"] Unable to collect flows for filter nic-2153554-eth1-vmware-sfw.2, rc=8
    17642026-01-18T08:41:25.425Znsx-exporter2105425NSX 2105425 - [nsx@6876 comp="nsx-esx" subcomp="agg-service" tid="2105488" level="WARNING"] Failed to get flow records: rc = 8
    17642026-01-18T08:41:25.428Znsx-exporter2105425NSX 2105425 - [nsx@6876 comp="nsx-esx" subcomp="agg-service" tid="2105488" level="WARNING"] Failed to get flow records: rc = 8

 

Environment

VMware NSX

Cause

  • The primary cause of the issue is an Out of Memory (OOM) condition affecting the nsx-exporter process.
  • The nsx-exporter process is responsible for monitoring NSX data, collecting statistics and monitoring DFW heap memory usage. and during high memory usage scenarios, the service fails due to the lack of available memory.

Resolution

Fixed in VMware NSX 4.2.1 and higher. See Download Broadcom products and software for steps to download this release.

Workaround:

Execute the commands below on the affected ESXi host using the root shell console:

  • For NSX version 4.1 or below:
    root# rm -f /var/core/*
  • For NSX version 4.1.1 or above:
    root# nsxcli -c del core-dump all

Run the following commands to check memory consumption by the exporter:

memstats -r group-stats -s name:min:max:consumed -u mb -g
vsish -e set /sched/groupPathNameToID host vim vmvisor nsx-exporter

The above commands help monitor the memory usage and determine if a restart is necessary.

If the nsx-exporter application has already crashed, it restarts automatically, so manual intervention is not required.

Restart the exporter if 'consumed' is nearing 'max' by executing the following command:

/etc/init.d/nsx-exporter restart
Note: Restarting the process may resolve the immediate memory exhaustion (OOM) issue, but this may not be a long-term solution.