How to use "/opt/likewise/bin/domainjoin-cli", CLI to handle Active Directory Domain Operations Join/Leave/Query on vCenter Server Appliance to resolve AD Account login failures with "Invalid Credentials" error message

Products

• VMware vCenter Server

Issue/Introduction

This article will help to resolve AD Domain login failure issues by performing Active Directory Domain Operations such as Join/Leave/Query on vCenter Server Appliance 6.x or 7.x using CLI,

Following are the operations that can be performed using the CLI:

• Join VCSA/PSC to AD Domain
• Leave VCSA/PSC from AD Domain
• Query Domain Join Status on VCSA/PSC

This article can also be utilized to perform AD Domain Join operation for the new vCenter Server Appliance.

Symptoms:

• Domain Account (AD login) fails with "Invalid Credentials" error message in vSphere Client
• SSO logs on vCenter Server or PSC shows errors similar to the one mentioned below :

Log files:
/var/log/vmware/sso/vmware-sts-idmd.log
OR
/var/log/vmware/sso/vmware-identity-sts-default.log
 

Note: The list of errors are not only the ones listed below and there could be more related error messages for the same issue
 

[<DATEandTIME> vsphere.local########-####-####-####-########9c78 INFO ] [VmEventAppender] EventLog: source=[VMware Identity Server], tenant=[vsphere.local], eventid=[USER_NAME_PWD_AUTH_FAILED], level=[ERROR], category=[VMEVENT_CATEGORY_IDM], text=[SimpleMessage[message=Failed to authenticate principal [account@domain_name]. Native platformerror [code: 851968][null][null]]], detailText=[Native platform error [code: 851968][null][null]], corelationId=[########-####-####-####-########9c78], timestamp=[1504459985968]
[<DATEandTIME> vsphere.local ########-####-####-####-########9c78 ERROR] [IdentityManager] Failed to authenticate principal [account@domain_name]. Native platform error [code: 851968][null][null]
com.vmware.identity.interop.idm.IdmNativeException: Native platform error [code: 851968][null][null]
at com.vmware.identity.interop.idm.LinuxIdmNativeAdapter.AuthenticateByPassword(LinuxIdmNativeAdapter.java:188) ~[vmware-identity-platform.jar:?]
at com.vmware.identity.idm.server.provider.activedirectory.ActiveDirectoryProvider.authenticate(ActiveDirectoryProvider.java:282) ~[vmware-identity-idm-server.jar:?]
at com.vmware.identity.idm.server.IdentityManager.authenticate(IdentityManager.java:2980) ~[vmware-identity-idm-server.jar:?]
at com.vmware.identity.idm.server.IdentityManager.authenticate(IdentityManager.java:9761) ~[vmware-identity-idm-server.jar:?]
 

[<DATEandTIME> vsphere.local ########-####-####-####-########9c78 INFO ] [IdentityManager] Authentication failed for user [account@domain_name] in tenant [vsphere.local] in [71] milliseconds with provider [domain_name] of type [com.vmware.identity.idm.server.provider.activedirectory.ActiveDirectoryProvider]
 

<DATEandTIME> vsphere.local        ########-####-####-####-########8e14 ERROR] [ServerUtils] Exception ‘com.vmware.identity.idm.IDMLoginException: Native platform error [code: -1765328360][null][null]’ com.vmware.identity.idm.IDMLoginException: Native platform error [code: -1765328360][null][null]

• Similar login failure can happen on vCenter Server in VMware Cloud Foundation environment as well

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on the environment.

Environment

VMware vCenter Server 7.0.x
VMware vCenter Server Appliance 6.5.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server Appliance 6.0.x

Resolution

If there is an AD Account login issue reporting "Invalid Credentials" for all the domain accounts, perform the below steps in order.

• Leave vCenter Server Appliance from Domain
• Join the vCenter Server Appliance to Domain
• Verify Domain Join Status from VCSA Command line
• Reboot the vCenter Server and retry login

Verify Domain Join Status from VCSA Command line:

1. Connect to the vCenter Server console or SSH session and log in using root credentials.
2. Run this command to query the domain join status of Appliance:
   
   /opt/likewise/bin/domainjoin-cli query

Leave vCenter Server Appliance from Domain:

Note: Custom permissions added in the vCenter Server Inventory will be lost if the user is from the Specific Domain which is configured to disjoin/remove and Identity Source is configured for that specific domain as Active Directory (Windows Integrated Authentication) option. Take necessary backups before trying Leave Domain operation.

1. Connect to the vCenter Server console or SSH session and log in using root credentials.
2. Run this command to disjoin the Appliance from the domain:
   
   /opt/likewise/bin/domainjoin-cli leave
   
   For example:
   
   /opt/likewise/bin/domainjoin-cli leave
   
   Include the domain name to the 'domain-join leave' syntax if the leave command fails. 
         Sample error message when removing the AD configuration from the host web client or in the vCenter Web UI:  "The user or group named '<domainName>\esx^admins' does not exist."
   
                  
   
        - Leave Command Syntax Including Domain Name:
         /opt/likewise/bin/domainjoin-cli leave <DomainName.com>
           Note: When the command is run, it requests the password for <domain.com>@<domainName> account. Use the password for the host's root ID' to complete this command.
    
3. Verify the status using "/opt/likewise/bin/domainjoin-cli query" command.
4. Run this command to restart the vCenter Server services:
   
   service-control --stop --all
   service-control --start --all

Joining vCenter Server Appliance to Domain:

1. Connect to the vCenter Server console or SSH session and log in using root credentials.
2. Run this command to join the Appliance to the domain:
   
   /opt/likewise/bin/domainjoin-cli join domain.com Domain_Administrator Password
   
   For example:
   
   /opt/likewise/bin/domainjoin-cli join vmware.local Administrator Passw0rd (Note: It will prompt for Password if password is not provided in the Command line)
    
3. Run this command to restart the vCenter services or reboot the VCSA:
   
   service-control --stop --all
   service-control --start --all

Additional Information

If the vCenter Server shows it is joined to the Active Directory domain both in the vSphere Client UI, there may be an issue the vCenter Server's computer object in Active Directory.   Delete that computer object before attempting to rejoin the Active Directory domain.

 

 

VMware Skyline Health Diagnostics for vSphere - FAQ (345059)

 

If /opt/likewise/bin/domainjoin-cli join fails with:  

 

Error: LW_ERROR_INVALID_MESSAGE [code 0x00009c46]
The Inter process message is invalid

 

Quickest resolution is to redeploy vCenter and restore from a VAMI-based backup.