VMware Directory Service (VMDIRD) going to unrecoverable state due to CEIP settings mismatch during vCenter Server deployment

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• VMDIRD is going to unrecoverable state post deploying a new vCenter with embedded PSC or a new external PSC, pointing to an existing vSphere domain in ELM (Enhanced Linked Mode)
• You will see entries similar to below in VMDIRD log

For Windows log location : %PROGRAMDATA%\VMware\vCenterServer\logs\vmdird\vmdir.log
For vCenter Server Appliance : /var/log/vmware/vmdird/vmdird-syslog.log

2019-07-25T14:24:57.790173+00:00 warning vmdird t@140488925480704: ReplDeleteEntry/VmDirInternalDeleteEntry: 66 (Operation not allowed on non-leaf). DN: cn=ConsentConfigContainer,cn=PhoneHome,cn=Services,dc=vsphere,dc=local, first attribute: cn, it's meta data: '59961:2:a1c1ae76-bb38-4414-9d56-164fe0554594:20190724200208.654:55550'. NOT resolving this possible replication CONFLICT. For this object, system may not converge. Partner USN 0
2019-07-25T14:24:58.094808+00:00 warning vmdird t@140488925480704: ReplDeleteEntry/VmDirInternalDeleteEntry: 66 (Operation not allowed on non-leaf). DN: cn=ConsentConfigContainer,cn=PhoneHome,cn=Services,dc=vsphere,dc=local, first attribute: cn, it's meta data: '60105:2:a1c1ae76-bb38-4414-9d56-164fe0554594:20190724200208.654:55550'. NOT resolving this possible replication CONFLICT. For this object, system may not converge. Partner USN 0
2019-07-25T14:24:59.912616+00:00 warning vmdird t@140488925480704: ReplDeleteEntry/VmDirInternalDeleteEntry: 66 (Operation not allowed on non-leaf). DN: cn=ConsentConfigContainer,cn=PhoneHome,cn=Services,dc=vsphere,dc=local, first attribute: cn, it's meta data: '61248:2:a1c1ae76-bb38-4414-9d56-164fe0554594:20190724200208.654:55550'. NOT resolving this possible replication CONFLICT. For this object, system may not converge. Partner USN 0
2019-07-25T14:25:06.605939+00:00 warning vmdird t@140488925480704: ReplDeleteEntry/VmDirInternalDeleteEntry: 66 (Operation not allowed on non-leaf). DN: cn=ConsentConfigContainer,cn=PhoneHome,cn=Services,dc=vsphere,dc=local, first attribute: cn, it's meta data: '65002:2:86bd65ad-2897-4e41-b201-80087e8c8c2a:20190724153438.014:55372'. NOT resolving this possible replication CONFLICT. For this object, system may not converge. Partner USN 0

Environment

VMware vCenter Server Appliance 6.7.x

Cause

This issue is caused due to Replication Conflicts of CEIP entries in VMDIRD Database

Resolution

This issue is resolved in vCenter Server 6.7 Update 3b, available at VMware Downloads.

Workaround:
To workaround this issue:

• Check the CEIP setting of the vCenter Server where new vCenter Server deployment is trying to point to in enhanced linked mode (replication partner)

• Redeploy the PSC/VCSA with the current CEIP state of existing vCenter Server

You may follow below steps to get the CEIP information of existing vCenter server:
 

Login to vCenter webclient as SSO administrator or with an account with Global Permissions
Navigate to Home -> Administration -> Customer Experience Improvement Program 

OR 

ssh to vCenter Server Appliance/ PSC and run below command
/usr/lib/vmware-cis-license/bin/ceip-cli.py get-ceip