ESXi host goes unresponsive after AD computer account is changed
search cancel

ESXi host goes unresponsive after AD computer account is changed

book

Article ID: 387873

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

In ESXi-/var/run/log/syslog.log file:

YYYY-MM-DDTHH:MM:SS Er(##) lwsmd[##]: [lsass] Failed to run provider specific request (request code = 14, provider = 'lsa-activedirectory-provider') -> error = #####, symbol = LW_ERROR_PASSWORD_MISMATCH, client pid = ###
YYYY-MM-DDTHH:MM:SS Wa(##) lwsmd[##]: [LwKrb5GetTgtImpl .. /lwadvapi/threaded/krbtgt.c:###] KRB5 Error code: -########## (Message: Preauthentication failed)

In ESXi-/var/run/log/vmkernel.log file:

YYYY-MM-DDTHH:MM:SS In(###) vmkernel: cpu#:###)UserWorld 'lwsmd' ### with cmdline '/usr/lib/vmware/likewise/sbin/lwsmd -- syslog', parent XXX
YYYY-MM-DDTHH:MM:SS In(###) vmkernel: cpu#:###)started from 'sh' ### with cmdline '/bin/sh /sbin/watchdog.sh -s lwsmd /usr/lib/vmware/likewise/sbin/lwsmd ++securitydom=lwsmdDom -- syslog', parent ###
YYYY-MM-DDTHH:MM:SS In(###) vmkernel: cpu#:###)started from 'init' ### with cmdline '/bin/init', parent 0
YYYY-MM-DDTHH:MM:SS In(###) vmkernel: cpu#:###)uw.### (####) requires #### KB, asked #### KB from likewise (###) which has ##### KB occupied and ### KB available.
YYYY-MM-DDTHH:MM:SS In(###) vmkernel: cpu#:###)Admission failure in path: host/vim/vmvisor/likewise:lwsmd.###:uw.###
YYYY-MM-DDTHH:MM:SS ALERT: hostd detected to be non-responsive

In ESXi-/var/run/log/vmkwarning.log file:

YYYY-MM-DDTHH:MM:SS Wa(180) vmkwarning: cpu##:########)WARNING: MemSchedAdmit: ####: Group likewise: Requested memory limit 0 KB insufficient to support effective reservation ##### KB

 

Environment

  • VMware vSphere ESXi 7.x
  • VMware vSphere ESXi 8.x

Cause

If the computer account password of an ESXi host is updated on AD, but doesn't get updated properly on ESXi, the old password can remain in the Likewise cache. This password is then repeatedly used by Likewise to attempt to authenticate to the AD domain. Eventually, the failed authentication attempts build up, likewise runs out of memory, and hostd goes unresponsive. The "LW_ERROR_PASSWORD_MISMATCH" indicates that the computer account password of the ESXi hosts was changed on the AD side, but wasn't updated on ESXi.

Resolution

Check the AD status of an ESXi host by running the command: /usr/lib/vmware/likewise/bin/lw-lsa get-status

Check if the host is showing the "LW_ERROR_PASSWORD_MISMATCH" error (see if the timestamps are recent, polls every 60s) using the command:
grep "LW_ERROR_PASSWORD_MISMATCH" /var/run/log/syslog.log

If either of the 2 above commands outputs issues (status not returning, or password mismatch error showing), perform the following steps on the host:

  1. Clear the cache & enum users again.
    /usr/lib/vmware/likewise/bin/lw-lsa ad-cache --delete-all
    /usr/lib/vmware/likewise/bin/lw-lsa enum-users
  2. Restart the likewise services on the host;
    /etc/init.d/lwsmd restart
  3. Remove the ESXi computer object from the AD domain, and attempt to re-add the host to the domain.

Additional Information

This KB is also applicable in scenarios where changes have been made to the AD computer account—not just in cases of password mismatches. Such modifications can lead the ESXi host to report lwsmd service errors and display Error: Not found when executing domain join query commands.

less vmkernel.all | grep -i "Admission failure in path: host/vim/vmvisor/likewise:lwsmd" -c
103353
less vmkernel.all | grep -i "started from 'init' 2097485 with cmdline" -c
115692
less vmkernel.all | grep -i "/bin/sh /sbin/watchdog.sh -s lwsmd" -c
104130

 

Powered by Wolken Software