High volume of ESXi logs sent to syslog server due to lwsmd memory leaks

Products

• VMware vSphere ESXi

Issue/Introduction

• ESXi hosts generate excessive log data, flooding the configured syslog server and rapidly rotating the local /var/run/log/vmkernel.log buffer. In severe cases, hundreds of thousands of log lines are generated within seconds.

• During the event, the /var/run/log/vmkernel.log file is continuously flooded with the following memory admission failure entries:
  

  YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu0:<PID>)Admission failure in path: host/vim/vmvisor/likewise:lwsmd.<PID>:uw.<PID>
  YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu0:<PID>)UserWorld 'lwsmd' <PID> with cmdline '/usr/lib/vmware/likewise/sbin/lwsmd --syslog', parent <PID>
  YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu0:<PID>)started from 'sh' <PID> with cmdline '/bin/sh /sbin/watchdog.sh -s lwsmd /usr/lib/vmware/likewise/sbin/lwsmd ++securitydom=lwsmdDom --syslog', parent <PID>
  YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu0:<PID>)started from 'init' <PID> with cmdline '/bin/init', parent 0
  YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu0:<PID>)uw.<PID> (<PID>) requires 1024 KB, asked 1024 KB from likewise (<PID>) which has 93112 KB occupied and 72 KB available.

Environment

VMware ESXi 8.x

Cause

• The issue is caused by memory leaks within the lwsmd (Likewise) process.
• Once the process exhausts its allocated memory limits, the memory scheduler repeatedly rejects new allocation requests, generating continuous "Admission failure" errors that flood the log buffers.

Resolution

Please subscribe to this kb article as the fix will be in the future ESXi patch release

Additional Information

• ESXi hosts intermittently disconnect from Active Directory or vCenter due to Likewise memory leak.
• ESXi hosts disjoins intermittently from Active Directory domain & the ESXi host shows not responding in the vCenter server