Error: The host has become unreachable. Remote logging to this host has stopped

Products

• VMware vSphere ESXi

Issue/Introduction

• ESXi hosts continuously report syslog-related errors in the vobd.log and host events. These errors are generated every few seconds, resulting in excessive log generation. Additionally, the ESXi hosts fail to forward logs to the configured remote syslog server, despite successful network connectivity and verification that the required ports are accessible.

• In ESXi /var/run/log/vobd.log, the following entries are observed:

[YYYY-MM-DD-THH:MM:SS] In(14) vobd[####]: [UserLevelCorrelator] ####us: [esx.problem.vmsyslogd.remote.failure] The host "[syslog server IP]:6514" has become unreachable. Remote logging to this host has stopped.
[YYYY-MM-DD-THH:MM:SS] In(14) vobd[####]: The event ([esx.problem.vmsyslogd.remote.failure] The host "[syslog server IP]:6514" has become unreachable. Remote logging to this host has stopped.) was sent immediately to hostd;
[YYYY-MM-DD-THH:MM:SS] In(14) vobd[####]: [GenericCorrelator] ####us: [vob.user.vmsyslogd.remote.failure] The host "[syslog server IP]:6514" has become unreachable. Remote logging to this host has stopped.

• In ESXi /var/log/.vmsyslogd.err, the following entries are observed:
  
  [YYYY-MM-DD-THH:MM:SS] vmsyslog.loggers.network : ERROR ] [syslog server IP]:6514 - socket error : [Errno 32] Broken pipe
[YYYY-MM-DD-THH:MM:SS] vmsyslog.loggers.network : ERROR ] Error shutting down socket.

Environment

VMware vSphere ESXi 8.x

Cause

The issue occurs because encryption is enabled on the remote syslog server, while the ESXi hosts continue to forward logs using unencrypted Transmission Control Protocol (TCP). This protocol mismatch causes the remote syslog server to reject the incoming connections, preventing successful log forwarding and resulting in repeated syslog-related errors on the ESXi hosts.

Resolution

To resolve this configuration mismatch, follow either of the two methods listed below. Performing these steps only affects the logging service and will not impact running virtual machines or host availability.

Method 1: Using the vSphere Client UI

1. Log in to the vSphere Client.
2. Navigate to Hosts and Clusters and select the affected ESXi host.
3. Click the Configure tab.
4. Under System, select Advanced System Settings > Edit.
5. Search for Syslog.global.logHost.
6. Change the value to include the ssl:// prefix (e.g., ssl://<syslog-server-ip>:6514).
7. Click OK.
8. Restart the syslog service under System > Services.

Method 2: Using the Command Line Interface (CLI)

1. SSH to the ESXi host as root.
2. Run: esxcli system syslog config set --loghost="ssl://<syslog-server-ip>:6514"
3. Run: esxcli system syslog reload
4. Verify logs in /var/log/.vmsyslogd.err.