Symptoms:
All authentication fails until reboot (likewise deadlock).
Authentication is intermittently slow due to likewise agents hanging for 1-5 minutes.
Root logins to the console or SSH may fail.
Open policy failures occur in the messages.log (/var/log/vmware/messages):
The vmware-identity-sts.log (/var/log/vmware/sso) stops logging until reboot. For example:
In the below reference, stsd stops logging at 2020-10-25T11:11:47.564Z and continues after reboot at 2020-10-25T12:09:13.074Z.
DCE/RPC code for likewise had a major update in VMware vCenter Server 6.7 GA onward. The updates resulted in lsass endpoints being corrupted during a string copy operation.
Endpoints are used when sending SMB2 CREATE request operations to domain controllers. The SMB2 CREATE request creates a file that is used to perform lsa_LookupNames2 and lsa_OpenPolicy2 requests that take place during user logins. When the corrupted endpoint is used (\\pipe\\lsassc) the domain controller will return an NTSTATUS of "STATUS_OBJECT_NAME_NOT_FOUND". This results in LW_ERROR_RPC_OPENPOLICY_FAILED failures in the /var/log/vmware/messages logs. A single open policy failure does not cause a login to fail. In larger environments, too many backed up lsa_LookupNames2 and lsa_OpenPolicy2 requests using the corrupted endpoint can cause likewise to intermittently hang (1-5 mins) or deadlock until reboot.
This issue is resolved in VMware vSphere vCenter Server 7.0 U2. To download go to the Patch Downloads page.
This issue is resolved in VMware vSphere vCenter Server 6.7 U3m. To download go to the Patch Downloads page.
Workaround:
Switch all identity sources to AD over LDAPS. This avoids using likewise for AD authentication.