[VMC on AWS] NSX Edge Datapath-Memory Pool Exhaustion Impacting VMs on L2E Segments

Products

• VMware HCX
• VMware NSX Networking
• VMware Cloud on AWS

Issue/Introduction

Explain how Datapath Memory-Pool exhaustion on an NSX edge can impact VM's on L2E extended segments regardless of which SDDC the VMs are on.

Symptoms:

• Mempool usage alerts on L2E Source SDDC's NSX manager

• Packet loss on VM's tied to L2E segments

• Wavefront showing packet drops due to memory on the Edge

When the Edge datapath's mempool usage is over 85% of capacity, the Edge will trigger a vmwNSXEdgeMpoolUsage event to show in the related logs as well as the NSX Manager.  

Example:
1  edge NSX 1895 FABRIC [nsx comp="nsx-edge" subcomp="datapathd.stats(stats9)" level="ERROR" eventId="vmwNSXEdgeMpoolUsage"] {"event_state":85,"event_external_reason":"Edge datapath memory usage is over 85%","event_src_comp_id":"<ID>","event_sources":{"id":"<ID>"}}

NSX Manager Alerts:


If Wavefront service is enabled then you may see below events:


 

 

Environment

VMware NSX-T

Cause

Datapath Memory pool exhaustion on the NSX Edge causes packet loss. VMs tied to L2E segments are impacted by issues seen on the source side NSX Edge due to the flow of traffic. This impact can extend to VMs running on the remote end of the HCX Site-Pair as the Source NSX Edge is unable to process the L2E packet, dropping it, before it can be sent to the remote end.

Resolution

To address the Edge Datapath-Memory Pool Exhaustion please increase Edge's resources on the source side.

Additional Information

https://vmc.techzone.vmware.com/resource/designlet-vmware-cloud-aws-management-cluster-planning#introduction

https://vmc.techzone.vmware.com/resource/designlet-vmware-cloud-aws-hcx-cloud-cloud-c2c-migration-between-sddcs#introduction