Enabling sub-NUMA clustering on a Bare Metal Edge may result in a shortage of heap memory.

Products

• VMware NSX

Issue/Introduction

Symptoms:

Enabling sub-NUMA clustering on a Bare Metal Edge may result in a shortage of heap memory.
Heap memory is allocated per NUMA node, and most memory pools for datapath are allocated on the first node. With sub-NUMA clustering, the total memory is divided among four NUMA nodes instead of two, limiting the available memory on the each node.

You may encounter the following messages in /var/log/syslog on your bare-metal Edge:

level="WARNING" eventState="On" eventFeatureName="edge_health" eventSev="warning" eventType="edge_datapath_mempool_high"] The datapath mempool usage for malloc_heap on Edge node 43f649fa-####-####-####-##########a5 has reached 100%, which is at or above the high threshold value of 85%.

datapathd 21058 stats tname="stats50" [ERROR] rte malloc_heap is exhausted (100% is used) errorCode="EDG0400711"

 

Environment

VMware NSX-T

Cause

Bare Metal Edge does not support sub-NUMA clustering. A maximum of two NUMA sockets is supported.

Resolution

To resolve the heap memory issues, you should disable sub-NUMA clustering.
This can be done in the BIOS settings. Please consult your server hardware vendor for instructions on how to change BIOS setting.
After disabling sub-NUMA clustering (which requires a reboot), there should be two expected heaps.

Additional Information

Impact/Risks:

sub-NUMA clustering on Bare Metal Edge results in shortage of heap memory.