vCenter services fail to start with error "ERROR_LOGON_FAILURE"

Products

• VMware SDDC Manager

Issue/Introduction

vCenter goes down after a restart/reboot of services and does not come up.

In vmon logs (/var/log/vmware/vmon/), the below errors are observed:

YYYY-MM-DDTHH:MM Wa(03)+ host-xxxx "args": [
YYYY-MM-DDTHH:MM Wa(03)+ host-xxxx "Error 46 while deleting svcaccount \"vpxd-svc-acct-xxxxxxxxx\":\ndir-cli failed. Error 1326: Operation failed with error ERROR_LOGON_FAILURE (1326)\n"
YYYY-MM-DDTHH:MM Wa(03)+ host-xxxx ],
YYYY-MM-DDTHH:MM Wa(03)+ host-xxxx "localized": "An error occurred while invoking external command : 'Error 46 while deleting svcaccount \"vpxd-svc-acct-xxxxxxxxx\":\ndir-cli failed. Error 1326: Operation failed with error ERROR_LOGON_FAILURE (1326) \n'"

 

Environment

VMware Cloud Foundation 4.x

VMware Cloud Foundation 5.x

VMware vCenter Server 7.x

VMware vCenter Server 8.x

Cause

This issue occurs if the vCenter machine account has a password length of more than 32 characters. 

Resolution

Please take backups of all vCenters before proceeding with the following steps.

Steps:

1. Power down all vCenters and take a snapshot of all vCenters in the ELM.

   • If the vCenter are standalone, please take a no-memory snapshot of the vCenter VM.

2. Power on the vCenters.

3. Expected Outcome:

   • Services will not start after powering on. This is expected behavior if the vCenters have 50-character passwords.

4. Change the password policy via command line (To be done on all vCenters if in ELM):

   a. Create the following files and save them:

   • mod_max.ldif

     dn: cn=password and lockout policy,dc=vsphere,dc=local
changetype: modify
replace: vmwPasswordMaxLength
vmwPasswordMaxLength: 20

   • mod_min.ldif

     dn: cn=password and lockout policy,dc=vsphere,dc=local
changetype: modify
replace: vmwPasswordMinLength
vmwPasswordMinLength: 8

   b. Run the following commands to apply the changes:

   /opt/likewise/bin/ldapmodify -h localhost -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -w ‘<admin-password>’ -f mod_min.ldif /opt/likewise/bin/ldapmodify -h localhost -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -w ‘<admin-password>’ -f mod_max.ldif

   c. Restart vmdird service:

   service-control --restart vmdird

   d. Verify the changes: Check the values of the vmwPasswordMaxLength and vmwPasswordMinLength attributes using the following command:

   /opt/likewise/bin/ldapsearch -o ldif-wrap=no -LLL -h localhost -b "cn=password and lockout policy,dc=vsphere,dc=local" -s sub -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -w ‘<admin-password>’

5. Once the password policy has been updated on all vCenters, you can reset the passwords on the affected vCenters using the reset machine password script.

6. Restart all services on all vCenter servers after resetting the passwords. Restarting VMware vCenter Server Appliance services

Additional Information

Suggested password policy:

Keep the "Minimum length" below 20 characters.

OR

Limit the value of "Maximum length" in the password policy to 32 and below when the "Minimum length" is set to above 20.