Datapathd 4761 firewalldp [ERROR] Memory resource from hugepage exhausted in the firewall service
search cancel

Datapathd 4761 firewalldp [ERROR] Memory resource from hugepage exhausted in the firewall service

book

Article ID: 413441

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • Gateway Firewall drops traffic during a burst in activity. There is an Edge Datapath Mempool Alarm around the same time.
  • Virtual machines may have unexplained connection interruptions 

Edge Health Alarm -  Edge Datpath Mempool High

Edge syslog (/var/log/syslog)

2025-09-03T15:19:26.084Z ############-edge03.########.com NSX 4761 FIREWALL [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="firewalldp" level="ERROR"]  message repeated 412 times: [Memory resource from hugepage exhausted in the firewall service size=1128(0M)]

2025-09-03T15:19:26.210Z ###########-edge03.########.com ########## #### - -  message repeated 77 times: [ 2025-09-03T15:19:26Z datapathd 4761 firewalldp [ERROR] Memory resource from hugepage exhausted in the firewall service size=1128(0M) ]

2025-09-03T15:19:25.423Z ##########-edge03.#########.com NSX 4761 FIREWALL [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="firewalldp" level="ERROR"] Dropped 826 log messages in last 23245 seconds (most recently, 23244 seconds ago) due to excessive rate

2025-09-03T15:19:26.002Z ###########-edge03.#######.com datapath-systemd-helper 4585 - -  2025-09-03T15:19:26Z datapathd 4761 firewalldp [ERROR] Memory resource from hugepage exhausted in the firewall service size=1128(0M)

2025-09-03T15:19:26.045Z ###########-edge03.#######.com 4a590c7a7618 3726 - -  2025-09-03T15:19:26Z datapathd 4761 firewalldp [ERROR] Memory resource from hugepage exhausted in the firewall service size=1128(0M)

Environment

VMware NSX

VMWare NSX-T Datacenter

Cause

Memory is exhausted on the Edge Node due to traffic burst.  During these bursts, there is inadequate memory for new flows, causing packets to drop.

In this case there is stateful default firewall enabled on Logical Routers and there appears to be a lot of hits from a burst of traffic that occurs on the Logical Router thus using up all the memory.

Resolution

Monitor the connections for the specific Logical Router and check if it's an expected traffic pattern.

Since the only rule is accept and if the traffic pattern of the burst is expected, the rule can be disabled on the firewall on the specific Logical Router in order not to see this memory alarm.

If the stateful firewall service is not needed

  • Disable the rule
  • Alternatively, replacing the rule with a stateless firewall rule will not have memory consumption for the states during the traffic bursts.

If the stateful firewall is needed

  • Edge Node will need to be resized appropriately to handle the traffic flows.

Additional Information

For collecting and analyzing DPDK Memory Usage open a ticket with Broadcom Support

Install the Python Script in this KB article:  Edge Datapath Stats Collection script for 3.x, 4.0 and 4.1

Upload the dp-stats.log files to your case.