Determining replication agreements and status with the Platform Services Controller (PSC)
search cancel

Determining replication agreements and status with the Platform Services Controller (PSC)

book

Article ID: 316504

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

This article provides information on using the vdcrepadmin command-line interface (CLI) for reviewing the existing vSphere domain, Platform Services Controllers (PSC) that make up your vSphere domain as well as checking the replication agreements configured and replication status within your environment.
Although the utility can be used for other operations, at this time, only what is documented must be executed by technical support staff and customers.
 
The vdcrepadmin CLI is located in these directories depending on the PSC distribution used:
  • Appliance: /usr/lib/vmware-vmdir/bin

    Note: This requires BASH Shell access to the appliance. For steps on switching from the appliance shell to the BASH shell, see Toggling the vCenter Server Appliance 6.x default shell (2100508).
     
  • Windows: "%VMWARE_CIS_HOME%"\vmdird\

    By default, environment variable will be: C:\Program Files\VMware\vCenter Server\

Environment

VMware vCenter Server 6.0.x
VMware vCenter Server 7.0.x
VMware vCenter Server 8.0.x

Resolution

Process to determine replication agreements and status with the Platform Services:

Use the below parameters using the vdcrepadmin command-line interface (CLI) for reviewing the existing vSphere domain to check replication agreements configured and replication status within your environment:

  • showservers - Displays all of the PSCs in a vSphere domain .
  • showpartners - Displays the current partnerships from a single PSC within a vSphere domain.
  • showpartnerstatus - Displays the current replication status of a PSC and any of the replication partners of the PSC.
  • createagreement   - Allows for creation of additional replication agreements between PSCs within a vSphere domain.
  • removeagreement - Allows for removal of additional replication agreements between PSCs within a vSphere domain.

Note: For vCenter 6.7 the showfederationstatus command was added to support additional debugging function.
 

Using the showservers parameter

  1. Make a remote access into the windows Platform Services Controller or SSH into the Platform Services Controller appliance.
  2. Run this command to change directories to the vmdird folder:

    Appliance: cd /usr/lib/vmware-vmdir/bin
    Windows: cd "%VMWARE_CIS_HOME%"\vmdird\
     
  3. Run this command to show all PSCs in the vSphere domain:
    vdcrepadmin -f showservers -h PSC_FQDN -u administrator -w Administrator_Password
     
Note: You may need to enter the administrator_password in between quotes. 
vdcrepadmin -f showservers -h PSC_FQDN -u administrator -w "Administrator_Password"
 
For example:
vdcrepadmin -f showservers -h psc1.example.local -u administrator -w VMw@re123
 

You see output similar to:
cn=psc1.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc2.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc4.example.local,cn=Servers,cn=East,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc3.example.local,cn=Servers,cn=East,cn=Sites,cn=Configuration,dc=vsphere,dc=local

From the output, you can determine the required information..

For example:
cn=psc1.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
 
The Platform Services Controller is named PSC1.example.local.
The Platform Services Controller is located in the home site within the vSphere domain.
 
To determine this information, review the full output:
The Platform Services Controllers PSC1.* and PSC2.* are located within the site Home.
The Platform Services Controllers PSC3.* and PSC4.* are located within the site East.
 

Using the showpartners parameter

  1. Make a remote connection to the Windows Platform Services Controller or SSH into the Platform Services Controller appliance.
  2. Run this command to change directories to the vmdird folder:
    Appliance: cd /usr/lib/vmware-vmdir/bin

    Windows: cd "%VMWARE_CIS_HOME%"\vmdird\
     
  3. Use this command to display the current partnership from the specified PSC:


Note: For 6.7 -->vdcrepadmin -f showfederationstatus-h localhost -u administrator -w Administrator_Password
vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password

For example:
vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123

This command provides PSC specified by -h parameter, partnership.

For example:
vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
ldap://psc2.example.local

  1. Use the PSC from Step 3 to map out the topology of the current vSphere domain by re-running this command against each of the PSCs in order to determine all of the partnerships.

    For example:
    Note: You can use the showservers parameter to get a list of all of the PSCs in the domain.
vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
ldap://psc2.example.local

vdcrepadmin -f showpartners -h psc2.example.local -u administrator -w VMw@re123
ldap://psc1.example.local
ldaps://psc3.example.local

vdcrepadmin -f showpartners -h psc3.example.local -u administrator -w VMw@re123
ldap://psc4.example.local
ldaps://psc2.example.local

vdcrepadmin -f showpartners -h psc4.example.local -u administrator -w VMw@re123
ldap://psc3.example.local
 
From this series of output, you can determine these information:
  • PSC1.* has a replication partnership with PSC2*
  • PSC2.* has a replication partnership with both PSC1.* and PSC3.*
  • PSC3.* has a replication partnership with both PSC2.* and PSC4.*
  • PSC4.* has a replication partnership with PSC3.*
You can see that this environment was installed in an in-line fashion, with each PSC installed against the previous PSC, rather than a hub-and-spoke fashion where all of the PSCs would terminate to a central PSC.
 

Using the showpartnerstatus parameter:

Note: This CLI is limited to execution only against the local PSC. Using the command to query the replication status from one PSC to a different PSC is not yet supported.
  1. Make a remote connection to the Windows Platform Services Controller or SSH into the Platform Services Controller appliance.
  2. Run this command to change directories to the vmdird folder:
    Appliance: cd /usr/lib/vmware-vmdir/bin
    Windows: cd "%VMWARE_CIS_HOME%"\vmdird\
     
  3. Use this command to display the current replication status of the PSC and its partner nodes:
    vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123

    This command displays the current replication partner of the PSC and also the current replication status between the two nodes.

    For example:
    psc3:~ # /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123
    Partner: psc4.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 9502
    Partner has seen my change number: 9502
    Partner is 0 changes behind.

    Partner: psc2.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 9502
    Partner has seen my change number: 9502
    Partner is 0 changes behind.
From this output, you can determine the following information for the current node (psc3.example.local):
There are two replication agreements, one with psc2.* and another with psc4.*
  • You are able to connect to the two partner PSCs and query their status
  • You are currently in sync with all replication partners within the environment
The current Update Sequence Number (USN) value for this PSC is 9502.
 
Repeat this operation on the other PSCs in the vSphere domain to check all replication status.
 
If the replication continues to fail on the customers environment, review the /var/log/vmware/vmdird/vmdird-syslog.log or "%VMWARE_LOG_DIR%"\vmdird\vmdird-syslog.log file for details. This provides all information related to replication status and the objects that are replicated.
 

Using the createagreement parameter

Note: This cannot be used to create replication agreements between disparate (separate) vSphere domains.
  1. Make a remote connection to the Windows Platform Services Controller or SSH into the Platform Services Controller appliance.
  2. Run this command to change directories to the VMDIRD folder:
    Appliance: cd /usr/lib/vmware-vmdir/bin
    Windows: cd "%VMWARE_CIS_HOME%"\vmdird\
     
  3. Use this command to display the current partnership from the specified PSC:
    vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123

    This provides PSC specified by -h parameter, partnership.

    For example:
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
    ldap://psc2.example.local
     
  4. Map out the topology of the current vSphere domain by re-running this command against each of the PSCs in order to determine all of the partnerships.

    For example:
    Note: You can use the showservers parameter to get a list of all of the PSCs in the domain.
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
    ldap://psc2.example.local

    vdcrepadmin -f showpartners -h psc2.example.local -u administrator -w VMw@re123
    ldap://psc1.example.local
    ldaps://psc3.example.local

    vdcrepadmin -f showpartners -h psc3.example.local -u administrator -w VMw@re123
    ldap://psc4.example.local
    ldaps://psc2.example.local

    vdcrepadmin -f showpartners -h psc4.example.local -u administrator -w VMw@re123
    ldap://psc3.example.local
     
  5. With your topology defined, we can now generate a new replication agreement to create a Ring Topology. Using the environment in this article as a model, we need to generate additional replication agreements between:
    PSC1.* and PSC4.*

    However, in larger environments with more PSCs or environment that have grown, replication agreements may need to be removed or adjusted accordingly. For the latter use case, consult the Adding Platform Services Controllers to grow your vSphere Domain and updating replication agreements in the Related Information section of this article for more guidance.
     
  6. Use the following command to create a new replication agreement between PSCs to generate a Ring Topology:
    vdcrepadmin -f createagreement -2 -h Source_PSC_FQDN -H New_PSC_FQDN_to_Replicate -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f createagreement -2 -h psc1.example.local -H psc4.example.local -u Administrator -w VMw@re123

 

NOTE: PSC Names in the syntax is case sensitive, vSphere Client shows the name in lower case. Be cautious in case sensitivity of PSC name
Use the following as a example as visual representation of the recommended outcome:


 

  1. After completion, repeat Step 4 to confirm that you have generated a Ring Topology.

    Note: Due to replication time, it may take a few seconds to minutes for a complete Ring topology to be configured.

Using the removeagreement parameter

Note: You cannot utilize this command to completely remove all replication partnerships from a PSC. At a minimum, a PSC requires a single replication agreement depending on your topology.
  1. Make a remote connection to the Windows Platform Services Controller or SSH into the Platform Services Controller appliance.
  2. Run this command to change directories to the vmdird folder:
    Appliance: cd /usr/lib/vmware-vmdir/bin

    Windows: cd "%VMWARE_CIS_HOME%"\vmdird\
     
  3. Use this command to display the current partnership from the specified PSC:
    vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123

    This provides PSC specified by -h parameter, partnership.

    For example:
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
    ldap://psc2.example.local
     
  4. Map out the topology of the current vSphere domain by re-running this command against each of the PSCs in order to determine all of the partnerships.

    For example:

    Note: You can use the showservers parameter to get a list of all of the PSCs in the domain. In the example below, we have a Ring topology.
     
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
    ldap://psc2.example.local
    ldaps://psc4.example.local
     
    vdcrepadmin -f showpartners -h psc2.example.local -u administrator -w VMw@re123
    ldap://psc1.example.local
    ldaps://psc3.example.local
     
    vdcrepadmin -f showpartners -h psc3.example.local -u administrator -w VMw@re123
    ldap://psc4.example.local
    ldaps://psc2.example.local
     
    vdcrepadmin -f showpartners -h psc4.example.local -u administrator -w VMw@re123
    ldap://psc3.example.local
    ldaps://psc1.example.local
  5. With your topology defined, we can now evaluate the removal of replication agreement. Using the environment in this article as a model, we can remove the replication agreements between in order to revert the four node topology into an Inline layout from its previous Ring layout:
    PSC1.* and PSC4.*
     
  6. Use the following command to remove an existing replication agreement between PSCs:
    vdcrepadmin -f removeagreement -2 -h Source_PSC_FQDN -H PSC_FQDN_to_Remove_from_Replication -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f removeagreement -2 -h psc1.example.local -H psc4.example.local -u Administrator -w VMw@re123

    This will revert the four node topology into an Inline layout from its previous Ring layout.
     
  7. Repeat this operation to remove any additional replication agreements between PSCs.

 

ELM.SH (An automated method for checking all the above commands in a single report.)

Running each vdcrepadmin command requires the user to re-enter the password. As opposed to entering in the password for each of these 6 commands, multiplied by the amount of vCenter's in a SSO domain. This script automates the checking of all replication commands as well as the VMDIRD state. This script makes no changes, but instead runs the above replication agreement commands to read the present status and health of the vCenter's in a given SSO domain.

Usage: Create a file on the vCenter (elm.sh) using vi elm.sh for example. Copy and paste the below bash script contents to the file and save with :wq! From there to run chmod  +x elm.sh and ./elm.sh to print the replication health report from elm.sh

If you see a bad interpreter message as a result of copying between Windows and Linux. You can use this command on the vCenter to correct the formatting to allow the script to run successfully: sed -i -e 's/\r$//' elm.sh

 

#!/bin/bash
read -p "Enter [email protected] password: " -s pw
clear
echo ">>>>>>>>>>>>>>>>>>>>>>"
echo "PNID"
/usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost
echo ">>>>>>>>>>>>>>>>>>>>>>"
echo "                      "
echo "########################################"
echo "ShowServers"
/usr/lib/vmware-vmdir/bin/vdcrepadmin -f showservers -h localhost -u administrator -w $pw
echo "                      "
echo "                      "
echo "########################################"
echo "ShowPartners"
/usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartners -h localhost -u administrator -w $pw
echo "                      "
echo "                      "
echo "########################################"
echo "ShowPartnerStatus"
/usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w $pw
echo "                      "
echo "                      "
echo "########################################"
echo "ShowFederationStatus"
/usr/lib/vmware-vmdir/bin/vdcrepadmin -f showfederationstatus -h localhost -u administrator -w $pw
echo "                      "
echo "                      "
echo "########################################"
echo "VMDIR State"
/usr/lib/vmware-vmafd/bin/dir-cli state get --password $pw

 

Additional Information

Verifying the replication after installing vCenter Server or creating a user/group

Using the showpartnerstatus CLI, you can monitor the current replication status across your environment. There are times in which replication may not be functioning properly and replication data from one PSC may not be reaching another PSC.

After modifying objects within the PSC, either via creation, deletion or actual modification, these changes are replicated to the PSCs after 30 seconds. When multiple PSCs are daisy chained together (in-line), this can result in a wave-replication in which PSCs in the chain will request an update, implement the update, and then its partner node will perform the same, eventually converge on the same replica data across the same vSphere domain.
 
After creating and deleting an object via the vSphere Web Client UI (user), you can observe these objects deletion permeate the domain:
  1. User creation and deletion on psc1.*
    psc1:~ # /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123
    Partner: psc2.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 12227
    Partner has seen my change number: 12222
    Partner is 5 changes behind.
     
  2. Repeating this command from the psc2.* simultaneously:

    Before modification from psc1.* is picked up
    psc2:~ # /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123
    Partner: psc1.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 14539
    Partner has seen my change number: 14539
    Partner is 0 changes behind.

    Partner: psc3.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 14534
    Partner has seen my change number: 14534
    Partner is 5 changes behind.

    After modification from psc1.* is picked up
    psc2:~ # /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123
    Partner: psc1.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 14539
    Partner has seen my change number: 14539
    Partner is 0 changes behind.

    Partner: psc3.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 14539
    Partner has seen my change number: 14539
    Partner is 0 changes behind.
     
  3. Repeating this command from the psc3.* simultaneously:

    Before modification from psc2.* is picked up
    psc3:~ # /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123
    Partner: psc4.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 9530
    Partner has seen my change number: 9523
    Partner is 7 changes behind.

    Partner: psc2.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 9530
    Partner has seen my change number: 9530
    Partner is 0 changes behind.

    After modification from psc2.* is picked up

    psc3:~ # /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartnerstatus -h localhost -u administrator -w VMw@re123
    Partner: psc4.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 9530
    Partner has seen my change number: 9530
    Partner is 0 changes behind.

    Partner: psc2.example.local
    Host available: Yes
    Status available: Yes
    My last change number: 9530
    Partner has seen my change number: 9530
    Partner is 0 changes behind.

Adding Platform Services Controllers to grow your vSphere Domain and updating replication agreements

Using the showservers , createagreements and removeagreements CLI, you can update your vSphere Domain's replication topology as addition Platform Services Controllers are added to your environment.
Using the model defined through out this article using a four node environment, we'll add an additional pair of Platform Services Controllers in a new site, bringing us to six total, and then update the replication topology.
  1. Make a remote access into the windows Platform Services Controller or SSH into the Platform Services Controller appliance.
  2. Run this command to change directories to the vmdird folder:
    Appliance: cd /usr/lib/vmware-vmdir/bin
    Windows: cd "%VMWARE_CIS_HOME%"\vmdird\
     
  3. Run this command to show all PSCs in the vSphere domain:
    vdcrepadmin -f showservers -h PSC_FQDN -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f showservers -h psc1.example.local -u administrator -w VMw@re123

    You see output similar to:
cn=psc1.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc2.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc4.example.local,cn=Servers,cn=East,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc3.example.local,cn=Servers,cn=East,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc5.example.local,cn=Servers,cn=West,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=psc6.example.local,cn=Servers,cn=West,cn=Sites,cn=Configuration,dc=vsphere,dc=local
From the output, you can determine the required information.
  • The Platform Services Controllers PSC1.* and PSC2.* are located within the site Home.
  • The Platform Services Controllers PSC3.* and PSC4.* are located within the site East.
  • The Platform Services Controllers PSC5.* and PSC6.* are located within the site West.
  1. Use this command to display the current partnership from the specified PSC:
    vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123

    This command provides PSC specified by -h parameter, partnership.

    For example:
    vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
    ldap://psc2.example.local
     
  2. Use the PSC from Step 4 to map out the topology of the current vSphere domain by re-running this command against each of the PSCs in order to determine all of the partnerships.

    For example:
    Note: You can use the showservers parameter to get a list of all of the PSCs in the domain.
vdcrepadmin -f showpartners -h psc1.example.local -u administrator -w VMw@re123
ldap://psc2.example.local
ldaps://psc4.example.local
 
vdcrepadmin -f showpartners -h psc2.example.local -u administrator -w VMw@re123
ldap://psc1.example.local
ldaps://psc3.example.local
 
vdcrepadmin -f showpartners -h psc3.example.local -u administrator -w VMw@re123
ldap://psc4.example.local
ldaps://psc2.example.local
 
vdcrepadmin -f showpartners -h psc4.example.local -u administrator -w VMw@re123
ldap://psc3.example.local
ldaps://psc1.example.local
ldaps://psc5.example.local
 
vdcrepadmin -f showpartners -h psc5.example.local -u administrator -w VMw@re123
ldap://psc4.example.local
ldaps://psc6.example.local
 
vdcrepadmin -f showpartners -h psc6.example.local -u administrator -w VMw@re123
ldap://psc5.example.local
From this series of output, you can determine these informtion:
  • PSC1.* has a replication partnership with both PSC2* and PSC4.*
  • PSC2.* has a replication partnership with both PSC1.* and PSC3.*
  • PSC3.* has a replication partnership with both PSC2.* and PSC4.*
  • PSC4.* has a replication partnership with both PSC3.* , PSC1.* and PSC5.*
  • PSC5.* has a replication partnership with both PSC4.* and PSC6.*
  • PSC6.* has a replication partnership with PSC5.*
  1. With your update topology defined, accounting for the addition of two new PSCs, we can now generate a new replication agreement to update the Ring Topology. Using the environment in this article as a model:

    We need to generate additional replication agreements between:
    PSC1.* and PSC6.*

    We need to remove the old replication agreements between:
    PSC1.* and PSC4.*
     
  2. Use the following command to create a new replication agreement between PSCs to update the Ring Topology:
    vdcrepadmin -f createagreement -2 -h Source_PSC_FQDN -H New_PSC_FQDN_to_Replicate -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f createagreement -2 -h psc1.example.local -H psc6.example.local -u Administrator -w VMw@re123
     
  3. Use the following command to remove an existing replication agreement between PSCs to :
    vdcrepadmin -f removeagreement -2 -h Source_PSC_FQDN -H PSC_FQDN_to_Remove_from_Replication -u administrator -w Administrator_Password

    For example:
    vdcrepadmin -f removeagreement -2 -h psc1.example.local -H psc4.example.local -u Administrator -w VMw@re123

    Use the following as a example as visual representation of the recommended outcome:

  1. After completion, repeat Step 4 to confirm that you have generated a Ring Topology with all 6 nodes.

    Note: Due to replication time, it may take a few seconds to minutes for a complete Ring topology to be configured.

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
When you run the above commands if you get the error "bash: vdcrepadmin: command not found", then try the below steps to address it: 

Once you cd into the right directory where vdcrepadmin is you can try to append ./ to the command so example: ./vdcrepadmin

Powered by Wolken Software