ESXi Host error due to vmsyslog logger lost log messages

Products

• VMware vSAN

Issue/Introduction

An ESXi host intermittently or persistently logs critical alerts in vmkernel.log indicating that syslog messages are being dropped:
ALERT: vmsyslog logger <IP address-##-##-##-01:514> lost #### log messages
ALERT: vmsyslog logger <IP address-##-##-##-02:514> lost #### log messages

When reviewing physical network interface card (NIC) statistics using esxcli network nic stats get -n vmnicX, the host exhibits a high number of dropped or missed packets, such as:

• Receive missed errors (e.g., on vmnic1#)

• Receive packets dropped (e.g., on vmnic#)

Environment

VMware vSAN 8.0

Cause

This error occurs because the ESXi logging daemon (vmsyslogd) cannot successfully transmit logs to the configured remote syslog servers (via TCP or UDP) and is forced to discard them. The underlying root cause is extreme network resource saturation and buffer starvation at the physical NIC and driver layer, driven by the following factors:

1. Lack of Traffic and Protocol Stack Isolation

All primary VMkernel adapters—vmk0 (Management), vmk1 (vMotion), and vmk2 (vSAN)—are bound to a single Distributed Virtual Switch (DVS) and share the exact same defaultTcpipStack. This monolithic network layout forces unrelated, high-volume traffic types to compete for the same kernel routing tables, memory buffers, and physical uplink bandwidth.

2. Loss of Physical Redundancy

If a redundant physical uplink dedicated to high-throughput traffic (such as vSAN) is disconnected or disabled, the remaining active uplink (e.g., vmnic3) becomes a single point of failure.

3. Traffic Saturation & Buffer Overflow

During peak write workloads, storage traffic completely consumes the single available interface.

Driver-Layer Drops: The physical NIC's Rx (Receive) rings become entirely overwhelmed, forcing the driver to drop packets.

Hardware Missed Errors: The onboard hardware buffer of the physical card fills up completely, dropping incoming packets before the ESXi hypervisor can process them.

Because the Management traffic shares this congested pathway without guaranteed bandwidth, syslog communication is starved, resulting in the vmsyslog logger lost log messages error.

Resolution

To resolve this issue, implement the following tactical actions and structural optimizations:

Step 1: Restore Network Redundancy (Immediate Action)
Reconnect and re-enable any disconnected physical network adapters/uplinks (especially those handling vSAN or storage traffic).

Impact: This instantly doubles the available bandwidth, restores load balancing, and relieves the severe throughput bottleneck on the saturated physical interfaces.

Step 2: Migrate vMotion to a Dedicated TCP/IP Stack
Isolate vMotion migration traffic to prevent it from exhausting system-wide memory buffers during VM migrations.

Log in to the vSphere Client.

Navigate to the ESXi host -> Configure -> Networking -> VMkernel adapters.

Select the vMotion adapter (vmk1) and click Edit.

Change the TCP/IP stack from Default TCP/IP stack to vMotion TCP/IP stack.

Reconfigure the IP and routing settings for the new stack if prompted.