SSH and Host Client connections fail to ESXi host

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Login to a ESXi host fails with the Host Client fails with the error:
  
  Invalid username/password
  
  
• SSH connection fails with the error:
  
  

  Access denied
  Using keyboard-interactive authentication.
  /usr/lib/vmware/misc/bin/tally_vob.sh failed: exit code 1

  
  
  
• In the /var/log/hostd.log file, there are entries similar to:
  

  ####-##-##T##:##:##:###Z verbose hostd[787B890] [Originator@6876 sub=PropertyProvider] RecordOp ASSIGN: guest, 3. Sent notification immediately.
  ####-##-##T##:##:##:###Z verbose hostd[787B890] [Originator@6876 sub=PropertyProvider] RecordOp ASSIGN: summary.guest, 3. Sent notification immediately.
  ####-##-##T##:##:##:###Z verbose hostd[B9C2B70] [Originator@6876 sub=PropertyProvider] RecordOp ASSIGN: guest.disk, 3. Sent notification immediately.
  ####-##-##T##:##:##:###Z verbose hostd[BE81B70] [Originator@6876 sub=PropertyProvider] RecordOp ASSIGN: guest.disk, 3. Sent notification immediately.
  pam_tally2(vmware-authd:auth): Tally overflowed for user root
  pam_tally2(vmware-authd:auth): user root (0) tally 65534, deny 5
  pam_exec(vmware-authd:auth): /usr/lib/vmware/misc/bin/tally_vob.sh failed: exit code 1
  pam_exec(vmware-authd:auth): conversation failed
  [LikewiseGetDomainJoinInfo:354] QueryInformation(): ERROR_FILE_NOT_FOUND (2/0): Rejected password for user root from IP
  ####-##-##T##:##:##:###Z verbose hostd[BE81B70] [Originator@6876 sub=PropertyProvider opID=1a76ea97] RecordOp REMOVE: latestPage[10], session[52465b7c-5326-b51d-b605-ed487d1ae203]5222e38f-7794-fd1e-e466-ced1434c0dfc. Applied change to temp map.

  
  
  

Environment

VMware vSphere ESXi 6.x

VMware vSphere ESXi 7.x

VMware vSphere ESXi 9.x

Cause

This is caused if there is any service is configured with ESXi root account credentials or if root account is locked out due to multiple login failure

Resolution

To resolve this issue:

• Verify tasks and events for failed events.
• Verify if any 3rd party plugins are configured with the ESXi host root credentials.
• Run this command to find users logged in as root:
  
  pam_tally2 --user=root
  
  
• Run this command to log out all users logged in as root:
  
  pam_tally2 --user=root --reset