vCenter Server Upgrade fails at pre-check with error 'VMDir is not in Normal State'

Products

• VMware vCenter Server

Issue/Introduction

Patching vCenter Server fails at pre-check with the error 'VMDir is not in normal state'.



VMDir status returns Normal while checking on vCenter.

PatchRunner.log - /var/log/vmware/applmgmt/PatchRunner.log

YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements INFO vmdir Validating the VMDir state
YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements INFO vmdir No replication partner found so considering as standalone
YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements INFO vmdir_validator Going to validate VMDir state
YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements INFO vmdir_validator Getting dir-cli path
YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements INFO vmdir_validator Executing the dir-cli command to get state
YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements ERROR vmdir_validator dir-cli failed. Error 382312694: Access denied, reason = rpc_s_auth_method (0x16c9a0f6).
YYYY-MM-DDTHH:MM:SSZ vmdir:CollectRequirements ERROR vmdir VMDir is not in correct state

vmdird-syslog.log - /var/log/vmware/vmdird/ vmdird-syslog.log 

YYYY-MM-DDTHH:MM:SSZ err vmdird  t@139644435928640: VdirPasswordFailEvent from user(cn=vc_fqdn,ou=domain controllers,dc=###,dc=###), error(0)()
YYYY-MM-DDTHH:MM:SSZ err vmdird  t@139644435928640: Srv_rpc_srp_verifier_verify_session failed, status (382312692)

Environment

VMware vCenter Server 7.0.x
VMware vCenter Server 8.0.x

Cause

• As part of the VMDir patch script, the below command is run to get the VMDir state.

/usr/lib/vmware-vmafd/bin/dir-cli state get --login --password --server-name --domain-name

• This command fails if there are invalid characters in "dcAccountPassword".
• Manually executing the command returns with the error "Access denied".

/usr/lib/vmware-vmafd/bin/dir-cli state get --login vcsa.vmware.com --password 'dcAccountPassword' --server-name 'vc_fqdn' --domain-name '#####'

Resolution

NOTE : Ensure to have a valid backup/snapshot of the vCenter Server.

Reset the machine account password for the vCenter to remove the invalid characters in "dcAccountPassword" using the article below - 

"LDAP Error Code 49"/Error (49) error in vmdird logs in vCenter Server

Additional Information

For a similar issue with  the same Error but different Resolution message involving VMdir in Standalone mode, see "Vmdir service is not in Normal state", vCenter Server upgrade pre-check fails due to VMDIR in Standalone mode