vCenter services fail to start after patching

Products

• VMware vCenter Server

Issue/Introduction

• After Patching vCenter, services on the vCenter will fail to start
• Reverting to backups taken before the update does not resolve the issue
• Vmware-sps and vmware-vpxd.svcs specifically will fail to start 
  
  
  
• Credentials errors in the vmdir.log

XXXX-XX-XXTXX:XX@xxxxxxxxxxxxxxxx::ERROR: VmDirSendLdapResult: Request (Bind), Error (LDAP_INVALID_CREDENTIALS(49)), Message ((49)(SASL step failed.)), (0) socket (xxx.x.x.x)

• Minimum and Maximum values for password length are set to 20+ and 50 respectively 
• /var/log/vmware/vmdird/vmdird-syslog.log

XXXX-XX-XXTXX:XX@xxxxxxxxxxxxxxxx:INFO: MOD 7,rep,vmwPasswordMaxLength: (50)

XXXX-XX-XXTXX:XX@xxxxxxxxxxxxxxxx:INFO: MOD 7,rep,vmwPasswordMinLength: (25)

Environment

vCenter 8.0 U3

Cause

The default SSO password length was edited and was set to above 20 characters.

This causes the machine account password to take the maximum character length that is specified in the policy

If this is above 32 characters this will cause services to fail

Resolution

Set the "Minimum length" to less then 20.(irrespective of whatever value is set in "Maximum length" field)

 

 

1. Have VAMI file based backups in place for all VCs in ELM
2. Power down all VCs and take a snapshot of all (This is done in order to have a faster recovery option as recovering from VAMI backups is time-consuming)
3. Services will not start post this step. (Expected as vCenters have 50 char passwords)
4. Change the password policy from command line
5. Create a file mod_min.ldif and mod_max.ldif with the following contents and save the file
   

   mod_max.ldif --->
   
   dn: cn=password and lockout policy,dc=vsphere,dc=local
   changetype: modify
   replace: vmwPasswordMaxLength
   vmwPasswordMaxLength: 20
   
   mod_min.ldif  ---->
   
   dn: cn=password and lockout policy,dc=vsphere,dc=local
   changetype: modify
   replace: vmwPasswordMinLength
   vmwPasswordMinLength: 8

   
   
   
6. Run the command:
   

   /opt/likewise/bin/ldapmodify -h localhost -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -w ‘<admin-password>’ -f <filename>.ldif

7. Restart vCenter services
   

    service-control --stop --all && service-control --start --all