ESX/ESXi hosts fail to authenticate against Active Directory

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• ESX/ESXi hosts stop authenticating to the domain
   
• Log files indicate that lsassd has stopped running
   
• In the messages.log file, you see entries similar to:
  
  Sep 2 15:03:20 ushtfvesx2eoeg lsassd[14294]: 0xf7fd5700:Unable to enumerate trusts for 'corp.local' domain because it is offline
  Sep 2 15:03:31 ushtfvesx2eoeg kernel: [ 3126.377377] lsassd[14296]: segfault at 0000000000000018 rip 00000000003b5af4 rsp 00000000f7dd4260 error 4
  Sep 2 15:20:50 ushtfvesx2eoeg lsassd[6271]: 0xf7fac700:Unable to enumerate trusts for 'corp.local' domain because it is offline
• In the /var/log/likewise.log, you see the entries similar to:
  
  <YYYY-MM-DD>T<time>:ERROR:0xffb48540:[LsaSrvInitAuthProviders() /build/mts/release/bora-2542417/likewise/esxi-esxi/src/linux/lsass/server/api/auth_provider.c:294] Failed to load provider [<null>] at [/lib/liblsass_auth_provider_ad.so] [error code:16406]
  20170516082753:INFO:[IPC] Starting server
  <YYYY-MM-DD>T<time>:DEBUG:0xffb48540:[LsaSrvVerifyNetLogonStatus() /build/mts/release/bora-2542417/likewise/esxi-esxi/src/linux/lsass/server/lsassd/libmain.c:311] Error code: 136 (symbol: ERROR_NOT_JOINED)</time></time>
   
• In the /var/log/hostd.log, you see the entries similar to:
  
  <YYYY-MM-DD>T<time> [33180B70 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
  DJRunJoinProcess: 0x80047: 0x3B - Unknown error
  Stack Trace:
  /build/mts/release/bora-2542417/likewise/esxi-esxi/src/linux/domainjoin/libdomainjoin/src/djauthinfo.c:872
  /build/mts/release/bora-2542417/likewise/esxi-esxi/src/linux/domainjoin/libdomainjoin/src/djauthinfo.c:1218
  <YYYY-MM-DD>T<time> [31FC2B70 error 'ActiveDirectoryAuthentication' opID=0687C30C-00001699-58-92 user=vpxuser] vmwauth Exception: Exception 0xffff0000: Unknown exception
  <YYYY-MM-DD>T<time> [31FC2B70 info 'Vimsvc.ha-eventmgr' opID=0687C30C-00001699-58-92 user=vpxuser] Event 14425 : Join domain failed.
  <YYYY-MM-DD>T<time> [31FC2B70 info 'Vimsvc.TaskManager' opID=0687C30C-00001699-58-92 user=vpxuser] Task Completed : haTask-ha-host-vim.host.ActiveDirectoryAuthentication.joinDomain-303211694 Status error</time></time></time></time>
   

Environment

VMware vSphere ESXi 5.5
VMware ESX 4.1.x
VMware ESX 4.0.x
VMware ESXi 4.1.x Embedded
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 5.0
VMware ESXi 4.0.x Installable

Cause

This issue occurs when lsassd service fails when authenticating to the domain.

Resolution

To resolve this issue:

Note: VMware recommends to put the host in maintenance mode before proceeding, so that the production virtual machines are not impacted in any way.

1. Connect to the ESX/ESXi host using SSH or through the console.
    
2. Run this command to stop the lsassd service:
   
   # /etc/init.d/lsassd stop
    
3. Copy the /etc/krb5.conf file from a working host to the host experiencing the issue.
    
4. Run this command to start lsassd service:
   
   # /etc/init.d/netlogond restart
   # /etc/init.d/lwiod restart
   # /etc/init.d/lsassd restart
   
   After a few minutes, the host starts communicating with the domain. In addition, the /etc/likewise/krb5-affinity file is populated with all KDCs.