ESXi host stops logging and eventually becomes unresponsive.
Article ID: 388922
Updated On:
Products
Issue/Introduction
- Post upgrade/update to ESXi 8.0.1 or later ESXi stops logging and eventually becomes unresponsive.
- The issue is observed in ESXi 7.0.1 or later as well.
- This issue is random in nature and there is no definitive way to identify which host is impacted by this issue.
- Hosts typically becomes unresponsive within 12 to 24 hours after logging stops.
- Logging is stopped across all the host logs like vmkernel.log, vobd.log, hostd.log, vmkwarning.log, vmksummary.log etc...
Environment
vSphere ESXi 7.X
vSphere ESXi 8.X
Cause
On reloading vmsyslogd a SIGHUP signal is sent to vmsyslogd. When multiple reload signal are sent with very short delay it could lead to a deadlock as the signal handler is not using async signal safe functions.
Resolution
Fix available in following release:
VMware vSphere 7.0 P10 - https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/7-0/release-notes/esxi-update-and-patch-release-notes/vsphere-esxi-70u3v-release-notes.html
VMware vSphere 8.0 P05 - https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/release-notes/esxi-update-and-patch-release-notes/vsphere-esxi-80u3e-release-notes.html
Workaround:
1. To workaround the issue before the host becomes unresponsive, execute pkill vmsyslogd. The watchdog will automatically restart the service, restoring functionality.
Note - If the Logging of the ESXi Host has already stopped which means that the ESXi host vmsyslogd is already in deadlock state, then the below command will not help recover the host.
pkill vmsyslogd2. If the host has already entered the unresponsive state, the only recovery option is to perform a reboot of the host.
Additional Information
Feedback
