NSX VIB uninstallation fails due to module memory leak

Products

• VMware NSX

Issue/Introduction

Symptoms:
- NSX VIB uninstallation is failing with logging indicating a module memory leak
 
- /var/log/esxupdate.log contains string like the following examples:
VMKMod_UnloadModule: Unloading module nsxt-ipfix-20598730 failed: Consumed resource count of module is not zero (bad0145) Cannot remove module nsxt-ipfix-20598730: Consumed resource count of module is not zero
OR
VMKMod_UnloadModule: Unloading module nsx-vdrb failed: Consumed resource count of module is not zero (bad0145) Cannot remove module nsx-vdrb: Consumed resource count of module is not zero
 
- vmkernel.log contains "Failed to unload module <module name>, since its consumed resource count is <number greater than 0>"
<timestamp> cpu38:5527410)ALERT: Heap: 2793: Non-empty heap (nsxt-ipfix) being destroyed (avail is 111776, should be 114080).
<timestamp> cpu38:5527410)ALERT: Mod: 5062: Failed to unload module nsxt-ipfix-20598730, since its consumed resource count is 14. Waiting...
<timestamp> cpu32:5527410)ALERT: Mod: 5091: Failed to unload module nsxt-ipfix-20598730, since its consumed resource count is 14. Giving up.

Environment

VMware NSX

Cause

The above logging is indicative of a memory leak for the given module. This sort of memory leak will prevent the module from being unloaded, and cause VIB uninstallation to fail.

Resolution

Resolution information will vary depending on module name and NSX version.

Collect host support bundle and vmkernel-zdump before rebooting host, and open Service Request with VMware for more information.

You can refer to the following link to collect the vmkernel-zdump. Generating a VMkernel zdump manually from a dump file in ESXi host

 

Workaround:
Reboot the affected host to free up the problematic memory allocation. After host reboot, reattempt NSX uninstallation.