ESXi host goes unresponsive after AD computer account is changed
search cancel

ESXi host goes unresponsive after AD computer account is changed

book

Article ID: 387873

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • If the computer account password of an ESXi host is updated on AD, but doesn't get updated properly on ESXi, the old password can remain in the Likewise cache. This password is then repeatedly used by likewise to attempt to authenticate to the AD domain. Eventually the failed authentication attempts build up, likewise runs out of memory, and hostd goes unresponsive.
  • In syslog.log file, you can see lines similar to: 

YYYY-MM-DDTHH:MM:SS Er(27) lwsmd[XX]: [lsass] Failed to run provider specific request (request code = 14, provider = 'lsa-activedirectory-provider') -> error = 40022, symbol = LW_ERROR_PASSWORD_MISMATCH, client pid = XXX
YYYY-MM-DDTHH:MM:SS Wa(28) lwsmd[XX]: [LwKrb5GetTgtImpl .. /lwadvapi/threaded/krbtgt.c:262] KRB5 Error code: -1765328360 (Message: Preauthentication failed)

  • In vmkernel.log file, you can see lines similar to:

YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu8:XXX)UserWorld 'lwsmd' XXX with cmdline '/usr/lib/vmware/likewise/sbin/lwsmd -- syslog', parent XXX
YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu8:XXX)started from 'sh' XXX with cmdline '/bin/sh /sbin/watchdog.sh -s lwsmd /usr/lib/vmware/likewise/sbin/lwsmd ++securitydom=lwsmdDom -- syslog', parent XXX
YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu8:XXX)started from 'init' XXX with cmdline '/bin/init', parent 0
YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu8:XXX)uw.XXX (7459) requires 1024 KB, asked 1024 KB from likewise (828) which has 93080 KB occupied and 104 KB available.
YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu8:XXX)Admission failure in path: host/vim/vmvisor/likewise:lwsmd.XXX:uw.XXX
YYYY-MM-DDTHH:MM:SS ALERT: hostd detected to be non-responsive

Environment

ESXi 7.0
ESXi 8.0

Cause

AD computer account password updated in AD but not in Likewise. The "LW_ERROR_PASSWORD_MISMATCH" indicates that the computer account password of the ESXi hosts was changed on the AD side but wasn't updated on ESXi.

Resolution

Check the AD status of an ESXi host by running;
$ /usr/lib/vmware/likewise/bin/lw-lsa get-status

Check if the host is showing the "LW_ERROR_PASSWORD_MISMATCH" error (see if the timestamps are recent, polls every 60s);
$ grep "LW_ERROR_PASSWORD_MISMATCH" /var/run/log/syslog.log


If either of the 2 above commands outputs issues (status not returning, or password mismatch error showing), perform the following steps on the host;

1. Clear the cache & enum users again
$ /usr/lib/vmware/likewise/bin/lw-lsa ad-cache --delete-all
$ /usr/lib/vmware/likewise/bin/lw-lsa enum-users

2. Restart the likewise services on the host;
$ /etc/init.d/lwsmd restart

3. Remove the ESXi computer object from the AD domain, and attempt to re-add the host to the domain

Additional Information

This KB is also applicable in scenarios where changes have been made to the AD computer account—not just in cases of password mismatches. Such modifications can lead the ESXi host to report lwsmd service errors and display Error: Not found when executing domain join query commands.

└─$ less vmkernel.all | grep -i "Admission failure in path: host/vim/vmvisor/likewise:lwsmd" -c
103353
 less vmkernel.all | grep -i "started from 'init' 2097485 with cmdline" -c
115692
─$ less vmkernel.all | grep -i "/bin/sh /sbin/watchdog.sh -s lwsmd" -c
104130

 

Powered by Wolken Software