Adding a host to Active Directory domain using vSphere Authentication Proxy(vmcam) fails intermittently with error code 41737

Products

• VMware

Issue/Introduction

To join the ESXi host to domain

Symptoms:

• Unable to join ESXi hosts using autodeploy  and  authentication proxy.
• It fails with an error similar to "The specified vSphere Authentication Proxy Server is not reachable, or has denied access to the service"
• During the second boot of ESXi host it can join to the domain.
• The ESXi  will show similar error as below during domain join.

 hostd.log
 2019-05-06T15:30:04.964Z verbose hostd[EB81B70] [Originator@6876 sub=PropertyProvider opID=b6644746 user=vpxuser] RecordOp ASSIGN: info, haTask--vim.InternalStatsCollector.queryLatestVmStats-183332122. Applied change to temp map.
LikewiseGetDaemonStatus: lwsmd is running
LsaSrvVerifyLwsmStatus: lwsmd ping successful
LikewiseStopService: stopping service: lsass
[LwSetupMachineSession:278] LwKrb5GetTgt(): 41737
[LikewiseJoinDomainWithMachineAccount:1015] LwSetupMachineSession(): 41737
2019-05-06T15:30:05.447Z error hostd[F5C6B70] [Originator@6876 sub=ActiveDirectoryAuthentication opID=AuthJoinDomainFormMediator-apply-225860-ngc:70022467-df-48-4741 user=vpxuser:Domain\User] vmwauth Exception: Exception 0xffff0000: Unknown exception
2019-05-06T15:30:05.448Z verbose hostd[F5C6B70] [Originator@6876 sub=PropertyProvider opID=AuthJoinDomainFormMediator-apply-225860-ngc:70022467-df-48-4741 user=vpxuser:AFI\ia-mparva] RecordOp REMOVE: latestPage[3661], session[52554650-c0e6-6118-7801-5b45e8162ce7]52d1f489-0bfd-ec51-ea81-cad163f84d35. Applied change to temp map.
2019-05-06T15:30:05.448Z verbose hostd[F5C6B70] [Originator@6876 sub=PropertyProvider opID=AuthJoinDomainFormMediator-apply-225860-ngc:70022467-df-48-4741 user=vpxuser:Domain\User] RecordOp ADD: latestPage[3671], session[52554650-c0e6-6118-7801-5b45e8162ce7]52d1f489-0bfd-ec51-ea81-cad163f84d35. Applied change to temp map.
2019-05-06T15:30:05.448Z verbose hostd[F5C6B70] [Originator@6876 sub=PropertyProvider opID=AuthJoinDomainFormMediator-apply-225860-ngc:70022467-df-48-4741 user=vpxuser:Domain\User] RecordOp ASSIGN: latestEvent, ha-eventmgr. Applied change to temp map.
2019-05-06T15:30:05.448Z info hostd[F5C6B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=AuthJoinDomainFormMediator-apply-225860-ngc:70022467-df-48-4741 user=vpxuser:Domain\User] Event 3671 : Join domain failed.
2019-05-06T15:30:05.448Z info hostd[F5C6B70] [Originator@6876 sub=Vimsvc.TaskManager opID=AuthJoinDomainFormMediator-apply-225860-ngc:70022467-df-48-4741 user=vpxuser:Domain\User] Task Completed : haTask-ha-host-vim.host.ActiveDirectoryAuthentication.joinDomainWithCAM-183332119 Status error

• When doing a packet capture we notice below

       Error: KRB5KDC_ERR_C_PRINCIPAL_UNKNOWN

Note:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Cause

 vCenter and ESXi  host is contacting different domain controllers of the AD domain. Due to this, if there is any replication delay between those two domain controllers,the ESXi  host might not be able to find the computer object created for it by the VC on the AD setup, thus resulting in LW_ERROR_KRB5KDC_ERR_C_PRINCIPAL_UNKNOWN error.

Resolution

This issue is resolved in ESXi 6.7 Patch ESXi670-201912001 (ESXi 6.7 P01) and in ESXi 6.5 Patch ESXi650-201912002 (ESXi 6.5 P04). To download go to  the Customer Connect Patch Downloads page

Refer to ESXi 6.7 patch  release notes   and ESXi 6.5 Patch release notes

Workaround:
Delay the domain join procedure post account creation. Rebooting the ESXi host adds delay and domain join works.

Additional Information

Ensure all permissions are setup correctly as per VMware KB :Joining vCenter Server Appliance or ESXi host into Active Directory domain fails with error: LW_ERROR_LDAP_CONSTRAINT_VIOLATION or LW_ERROR_LDAP_INSUFFICIENT_ACCESS

Impact/Risks:
No Impact