'Unable to retrieve pairs from extension server' errors are observed for vSphere Replication on the site recovery client.
search cancel

'Unable to retrieve pairs from extension server' errors are observed for vSphere Replication on the site recovery client.

book

Article ID: 404218

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

  • The vcenter servers are in enhanced linked mode

  • On the vSphere Client, under site recovery plugin details you can see that the vSphere Replication appliance reports as 'Not Configured' and the information states that the VRMS solution user is missing or outdated

  • On launching the site recovery client from the remote vcenter server, we can see that it reports that it is unable to connect to the HBR management server and we cannot retrieve the extension pairs

  • When we try to reconfigure the appliance, it fails to register with the vcenter server

  • From the vSphere Replication appliance /opt/vmware/support/logs/dr/drconfig.log, we can see that the reason for the reconfigure failures are sso faults

    2025-07-11T11:18:50.262+05:30 info drconfig[01167] [SRM@6876 sub=DrConfigConfigurationManager opID=e6e28641-aa40-4864-8825-bbc61837a636-configure:29d2] ConfigureVRMS: Starting configuration task
    2025-07-11T11:19:09.330+05:30 error drconfig[01176] [SRM@6876 sub=ConfigureVrmsOp opID=e6e28641-aa40-4864-8825-bbc61837a636-configure:29d2] command:
    ................................
    --> stderr:
    --> /usr/lib/python3.10/getpass.py:91: GetPassWarning: Can not control echo on the terminal.
    -->   passwd = fallback_getpass(prompt, stream)
    --> Warning: Password input may be echoed.
    --> Enter ssopassword:
    --> NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
    --> Unhandled exception
    --> (sso.fault.InternalFault) 
    2025-07-11T11:19:09.331+05:30 error drconfig[01176] [SRM@6876 sub=ConfigureVrmsOp opID=e6e28641-aa40-4864-8825-bbc61837a636-configure:29d2] Operation failed
    --> (vmodl.fault.SystemError) {
    -->    faultCause = (vmodl.MethodFault) null,
    -->    faultMessage = <unset>,
    -->    reason = "Failed to register VRMS."
    -->    msg = ""
    --> }

  • Inaddition, to this, from the /opt/vmware/hms/logs/hms.log we can see repeated events stating 'Provided credentials are not valid'

    2025-07-11T11:14:47.470+05:30 warning drconfig[24595] [SRM@6876 sub=SupportBundleRequestHandler.HmsHealthHandler opID=812ec3ad] HMS Health err output:
    --> com.vmware.jvsl.sso.SsoException: com.vmware.vim.sso.client.exception.AuthenticationFailedException: Provided credentials are not valid.
    -->     at com.vmware.jvsl.sso.ServiceAccountStsService.getToken(ServiceAccountStsService.java:132)
    -->     at com.vmware.hms.apps.util.ApplianceConfigurationTest.checkSsoUser(ApplianceConfigurationTest.java:224)
    -->     at com.vmware.hms.apps.util.ApplianceConfigurationTest.verifyCurrentConfig(ApplianceConfigurationTest.java:203)
    -->     at com.vmware.hms.apps.util.ApplianceConfigurationTest.run(ApplianceConfigurationTest.java:146)
    -->     at com.vmware.hms.apps.util.App.run(App.java:103)
    -->     at com.vmware.hms.apps.util.App$1.run(App.java:152)
    -->     at com.vmware.jvsl.run.ExceptionHandlerRunnable$1.run(ExceptionHandlerRunnable.java:47)
    -->     at com.vmware.jvsl.run.CheckedRunnable.withoutChecked(CheckedRunnable.java:19)
    -->     at com.vmware.jvsl.run.ExceptionHandlerRunnable.withExceptionHandler(ExceptionHandlerRunnable.java:43)
    -->     at com.vmware.hms.apps.util.App.main(App.java:149)
    --> Caused by: com.vmware.vim.sso.client.exception.AuthenticationFailedException: Provided credentials are not valid.
    -->     at com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl

Environment

vSphere Replication 8.x

vSphere Replication 9.x

Cause

The vSphere Replication appliance is unable to register with its local vcenter server as the vmdir on the local vcenter server is in read only mode. Since the vSphere Replication appliance on one of sites is failing to register with its vcenter server, the site pair is reporting the error that it is unable to retrieve the extension pairs

Cause Justification

From the vcenter server var/log/vmware/vmdird/vmdird-syslog.log we can see below events

2025-07-11T06:00:02.756520+00:00 err vmdird  t@140687509001984: SASLSessionStep: sasl error (-13)(SASL(-13): authentication failure: client evidence does not match what we calculated. Probably a password error)
2025-07-11T06:00:02.756852+00:00 err vmdird  t@140687509001984: VmDirSendLdapResult: Request (Bind), Error (LDAP_INVALID_CREDENTIALS(49)), Message ((49)(SASL step failed.)), (0) socket (172.#.#.#)
2025-07-11T06:11:15.410467+00:00 err vmdird  t@140015615055616: User account control - (cn=com.vmware.vr-sa-d050350c-6733-4b78-9c9f-8f2a738d8772,cn=serviceprincipals,dc=vsphere,dc=local): (800000) flag set, new value=(800000) failed
2025-07-11T06:11:15.410496+00:00 warning vmdird  t@140015615055616: LoginBlocked DN (cn=com.vmware.vr-sa-d050350c-6733-4b78-9c9f-8f2a738d8772,cn=serviceprincipals,dc=vsphere,dc=local), error (9239)()
2025-07-11T06:11:15.410521+00:00 info vmdird  t@140015615055616: Bind failed () (9239)
2025-07-11T06:11:15.410545+00:00 err vmdird  t@140with 015615055616: VmDirSendLdapResult: Request (Bind), Error (LDAP_INVALID_CREDENTIALS(49)), Message (), (0) socket (127.0.0.1)
2025-07-11T06:11:15.410569+00:00 err vmdird  t@140015615055616: Bind Request Failed (127.0.0.1) error 49: Protocol version: 3, Bind DN: "CN=com.vmware.vr-sa-d050350c-6733-4b78-9c9f-8f2a738d8772,cn=ServicePrincipals,dc=vsphere,dc=local", Method: SASL
2025-07-11T06:11:15.538846+00:00 err vmdird  t@140015615055616: InternalModifyEntry: VdirExecutePostModifyCommitPlugins - code(9114)

Though with the above error messages it seems like the outdated password is causing the issue, the issue here is not because of the invalid password.

On checking further, we can also see events stating Error (LDAP_UNWILLING_TO_PERFORM(53)) indicating the vmdir could be in read only mode 

2025-07-11T06:00:02 err vmdird t@139743522182912: VmDirSendLdapResult: Request (Modify), Error (LDAP_UNWILLING_TO_PERFORM(53)), Message (Server in read-only mode), (0) socket (127.0.0.1)

On checking the vmdir state on the vCenter server we can see that the vmDir is in readOnly mode

/usr/lib/vmware-vmdir/bin/vdcadmintool

Please select:
0. exit
1. Test LDAP connectivity
2. Force start replication cycle
3. Reset account password
4. Set log level and mask
5. Set vmdir state
6. Get vmdir state
7. Get vmdir log level and mask

6

VmDir State is - Read only

Resolution

To resolve the vSphere Replication appliance configuration and site pair issues, we need to resolve the issues with vmdir database on the vcenter server. 

Please follow the steps documented in the below article to resolve the issues with the vcenter server

Fix PSC/vmdir inconsistencies using fixpsc python script

Once the vcenter server issue is resolved, reconfigure the VR appliance.

Powered by Wolken Software