NSX-v Edge memory usage grows continuously over time

Products

• VMware NSX for vSphere

Issue/Introduction

• Memory usage of Edge VM shows slow growth over many months, possibly more that one year.
• Using the command "show process monitor" from the Edge CLI, or top from root shell, viewing the process memory usage shows that the vmtoolsd process is using most of the memory.
• If left for long enough, an OOM condition will happen and the kernel will choose a process to terminate. Soon after, the Edge VM will also reboot.
• Both Edge HA peers will exhibit the same symptoms and OOM/reboot behavior within a few days of each other.
• When an OOM occurs, in the /var/log/message file, you see entries similar to: 
  
  <Date>T<Time> <Edge Hostname> kernel[]: [default]:  [kern.warning] syslog-ng invoked oom-killer: gfp_mask=0x, order=0, oom_score_adj=0
  
  Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX for vSphere 6.4.x

Cause

This issue occurs due to a slow memory leak in vmtoolsd version 10.0.5 which runs in the NSX-v Edge.

Resolution

This issue is resolved in VMware NSX for vSphere 6.4.8, available at Broadcom downloads.