/var/run/log/hostd.log
, the following messages are noticed.[YYYY-MM-DDTHH:MM:SS] In(166) Hostd[2098727]: [Originator@6876 sub=Vimsvc.ha-eventmgr opID=ad545190 sid=52687575] Event 1488 : Cannot login user [email protected]: no permission
[YYYY-MM-DDTHH:MM:SS] In(166) Hostd[2098715]: [Originator@6876 sub=Solo.Vmomi] Activation finished; <<52687575-9d5b-c00e-1e7d-1c2d6ed5ad1e, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : 35710'>>, ha-sessionmgr, vim.SessionManager.login, <vim.version.v8_0_3_0, internal, 8.0.3.0>, [N11HostdCommon18VmomiAdapterServer19ActivationResponderE:0x0000002569ef9548]>
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098715]: [Originator@6876 sub=Solo.Vmomi] Arg userName:
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098681]: --> "local-root"
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098715]: [Originator@6876 sub=Solo.Vmomi] Arg password:
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098681]: --> (not shown)
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098681]: -->
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098715]: [Originator@6876 sub=Solo.Vmomi] Arg locale:
[YYYY-MM-DDTHH:MM:SS] Db(167) Hostd[2098681]: --> "en"
[YYYY-MM-DDTHH:MM:SS] In(166) Hostd[2098715]: [Originator@6876 sub=Solo.Vmomi] Throw vim.fault.NoPermission
[YYYY-MM-DDTHH:MM:SS] In(166) Hostd[2098715]: [Originator@6876 sub=Solo.Vmomi] Result:
[YYYY-MM-DDTHH:MM:SS] In(166) Hostd[2098681]: --> (vim.fault.NoPermission)
# cat /var/run/vmware/watchdog/healthd
PID=525564
NRESTART=0
MAXRESTART=1000000
TSTAMP=1725485057
NQUICK=0
TQUICK=60
MAXQUICK=5
IMMORTAL=0
ARG=/usr/lib/vmware/healthd/bin/healthd
ARG=++securitydom=healthdDom,group=healthd
ARG=-r
ARG=/var/lib/vmware/osdata/healthd
ENV=USER=root <========= Uses root user
ENV=SHLVL=2
ENV=HOME=/
ENV=TERM=vt102
ENV=PATH=/bin
ENV=SHELL=/bin/sh
ENV=PWD=/
RLIMITS=-1,524288,256,4096
ps -cCJ
command confirms that the healthd runs every 5 minutes. # while true; do date >> /tmp/ps_CcJ.txt && ps -CcJ >> /tmp/ps_CcJ.txt; sleep 1; done
We can see the below entries in tmp/ps_CcJ.txt file in interval of 5 minutes.Example output of the healthd.log on ESXi:
2024-12-09T16:30:30.196Z In(166) healthd[2102511]: [Originator@6876 sub=PluginLauncher] Launching binary: /usr/lib/vmware/healthd/plugins/bin/vsan_health ++group=healthd-plugins,mem=40 -u http://!vmwLocalSocketHealthd
2024-12-09T16:30:30.370Z In(166) healthd[2102511]: [Originator@6876 sub=HealthdHandler] Report Path: /var/lib/vmware/osdata/healthd/vmw.vSANStatus
--
2024-12-09T16:35:30.199Z In(166) healthd[2100119]: [Originator@6876 sub=PluginLauncher] Launching binary: /usr/lib/vmware/healthd/plugins/bin/vsan_health ++group=healthd-plugins,mem=40 -u http://!vmwLocalSocketHealthd
2024-12-09T16:35:30.368Z In(166) healthd[2102511]: [Originator@6876 sub=HealthdHandler] Report Path: /var/lib/vmware/osdata/healthd/vmw.vSANStatus
--
2024-12-09T16:40:30.203Z In(166) healthd[2100130]: [Originator@6876 sub=PluginLauncher] Launching binary: /usr/lib/vmware/healthd/plugins/bin/vsan_health ++group=healthd-plugins,mem=40 -u http://!vmwLocalSocketHealthd
2024-12-09T16:40:30.369Z In(166) healthd[2102511]: [Originator@6876 sub=HealthdHandler] Report Path: /var/lib/vmware/osdata/healthd/vmw.vSANStatus
--
2024-12-09T16:45:30.206Z In(166) healthd[2100119]: [Originator@6876 sub=PluginLauncher] Launching binary: /usr/lib/vmware/healthd/plugins/bin/vsan_health ++group=healthd-plugins,mem=40 -u http://!vmwLocalSocketHealthd
2024-12-09T16:45:30.373Z In(166) healthd[2100130]: [Originator@6876 sub=HealthdHandler] Report Path: /var/lib/vmware/osdata/healthd/vmw.vSANStatus
--
2024-12-09T16:50:30.209Z In(166) healthd[2100114]: [Originator@6876 sub=PluginLauncher] Launching binary: /usr/lib/vmware/healthd/plugins/bin/vsan_health ++group=healthd-plugins,mem=40 -u http://!vmwLocalSocketHealthd
2024-12-09T16:50:30.374Z In(166) healthd[2100111]: [Originator@6876 sub=HealthdHandler] Report Path: /var/lib/vmware/osdata/healthd/vmw.vSANStatus
--
2024-12-09T16:55:30.212Z In(166) healthd[2100114]: [Originator@6876 sub=PluginLauncher] Launching binary: /usr/lib/vmware/healthd/plugins/bin/vsan_health ++group=healthd-plugins,mem=40 -u http://!vmwLocalSocketHealthd
2024-12-09T16:55:30.397Z In(166) healthd[2100114]: [Originator@6876 sub=HealthdHandler] Report Path: /var/lib/vmware/osdata/healthd/vmw.vSANStatus
VMware vSphere ESXi 8.0U3
vsan_health is a plugin launched every 5 minutes to capture the vSANmgmt daemon health. The plugin tries to log in to Hostd as a root user to get vsan stubs. When the host is in lockdown mode, the "root" user will be disabled. Hence we get the error in hostd.log
Broadcom has addressed the issue and a fix is scheduled to be in a future release of ESXi 8.0.
Please follow the below workaround as a temporary correction until fix has been released.
Workaround:
configstorecli config current set -c esx -g health -k vsan_health --path "enabled" --value False
/etc/init.d/health restart
Disabling this `vsan_health` plugin and restarting healthd, this error(s) should not come in hostd.log.