Enhanced Linked Mode (ELM) remains broken between the VCSA nodes

Products

• VMware vCenter Server
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

ELM may become "broken" due to many reasons, see some of these below -

• No offline snapshot was completed, and on restore, ELM is not in sync.
• Only 1 vCenter is restored to Snapshot/backup 
• Changes made to the networking of vCenter/s may disrupt replication

 

• In vCenter logs may see logs similar to the below: 

/var/log/vmware/vmdird/vmdird-syslog.log 

yyyy-mm-ddThh:mm:ss.Z.err vmdird  t@140008241473280: VmDirSendLdapResult: Request (Search), Error (LDAP_UNWILLING_TO_PERFORM(53)), Message (Server in not in normal mode, not allowing outward replication.), (0) socket (10.10.10.10)

• After setting the VMDIR state back to normal, may experience issues with machine/computer account authentication errors; may see similar errors to those below -
• /var/log/vmware/vmdird/vmdird-syslog.log 

yyyy-mm-ddThh:mm:ss.Z err vmdird t@123456789123: Bind Request Failed (x.x.x.x) error 49: Protocol version: 3, Bind DN: ..., Method: SASL 12024-08-02T13:52:12.965554-05:00 err vmdird t@140245530842880: SASLSessionStep: sasl error (-13)(SASL(-13): authentication failure: client evidence does not match what we calculated. Probably a password error)

Environment

VMware vCenter Server 

 

Cause

• Due to the VMDIR state being in Read-only, replication is not functioning in the environment 
• This may also be due to changes made in the environment which caused mismatches in machine/computer account passwords between the vCenter server nodes in the SSO domain 

Resolution

To resolve the issue, please apply below steps: 

Note - Ensure to have valid offline snapshots and/or file-based backups completed of all nodes in the SSO domain before making any changes

Confirm the partner status of the vCenters in linked mode with the following command: 
 /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator 

Resolution Steps: 

1. Set  VMDIR state back to normal with the below steps:

   • SSH into the vCenter with root credentials and run the following commands: 

              usr/lib/vmware-vmdir/bin/vdcadmintool  

              Then use option 5 to set the vmdir state to NORMAL

   • If the above fails proceed to step 2. 
2. Run the FixPSC script in KB: Fix PSC/vmdir inconsistencies using fixpsc python script
3. If there are still VMDIR inconsistencies please contact Broadcom Support for further assistance.