vSAN Witness Appliance Fails to Send Traffic to Remote Syslog Server

Products

• VMware vSphere ESXi
• VMware vSAN

Issue/Introduction

• The vSAN witness appliance fails to forward log traffic to a configured remote syslog server.
• This issue is typically observed when the witness appliance is correctly configured with a remote log host, but no logs are received at the destination.
• You may observe "lost log message" alerts in the UI or find frequent entries in the dropped log file located at /var/run/log/vmsyslogd-dropped.log.
  

  <Time stamp> Al(177) vmkalert: cpu3:#####)ALERT: vmsyslog logger <Syslog server FQDN>:514 lost #### log messages

Environment

VMware ESXi (vSAN Witness Appliance)
vSAN 7.x, 8.x

 

Cause

The ESXi firewall on the witness appliance is blocking outbound traffic for the syslog service. By default, the syslog ruleset may not be enabled, preventing communication over UDP 514, TCP 514, or TCP 1514.

Resolution

To resolve this issue, you must manually enable the syslog firewall ruleset on the vSAN witness appliance via the ESXi command line.

• Log in to the vSAN witness appliance via SSH or the ESXi Shell.
• Verify the current status of the syslog firewall ruleset:

  esxcli network firewall ruleset list --ruleset-id=syslog

  Name    Enabled  Enable/Disable configurable  Allowed IP configurable
  ------  -------  ---------------------------  -----------------------
  syslog    false                         true                     true

• Enable the syslog ruleset to allow outbound traffic:

  esxcli network firewall ruleset set --ruleset-id=syslog --enabled=true

  Name    Enabled  Enable/Disable configurable  Allowed IP configurable
  ------  -------  ---------------------------  -----------------------
  syslog     true                         true                     true

• Refresh the firewall configuration to apply the changes:

  esxcli network firewall refresh

• Verify connectivity to the remote syslog server using the network checker:

  nc -zu <syslog-server-ip> 514

Additional Information

• For further details on Configuring syslog on ESXi
• If logs continue to be dropped after enabling the firewall, check for network-layer blocking between the witness site and the syslog destination.