Site Recovery Manager or vSphere Replication cannot complete a site pair operation. The received single sign-on token is valid from XX to YY

Products

• VMware Live Recovery
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

1. When we login to remote site vcenter within SRM UI, an error is displayed:

"Failed trying to retrieve token: ns0:RequestFailed: EndTime: Thu Nov 14 11:16:07 CST 2019 is not after startTime: Thu Jan 09 15:36:50 CST 2020". The date is probably different.

2. SRM UI displays the error below when trying to pair sites: 

ERROR
Operation Failed
SRM server 'Memphis' cannot complete a pair operation. The received single sign-on token is valid from '2024-03-15 14:35:18.862' to '2024-03-15 22:35:18.862'. It is currently '2024-03-15 14:34:07.285'. The tolerance is 30000 milliseconds.
Operation ID: 32662462-####-####-####-##########1e
3/15/24, 9:35:19 PM +0700

3. From vCenter, vmware-identity-sts.log on remote vCenter, below log messages can be seen:

[2020-01-09T15:36:50.531+08:00 tomcat-http--43 vsphere.prd 8ac53787-####-####-####-##########47 DEBUG com.vmware.identity.sts.impl.HoKConditionsAnalyzer] Found HoK certificate [
[
  Version: V1
  Subject: OU=Site Recovery Manager client, O=VMware vSphere Client, C=US
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

  Key: Sun RSA public key, 2048 bits
  modulus: 26584583412174822090806596074634626235292534271726185542202376482549797065149045088447816850404034771759725273421613115401514482358184897091417150792312990801617108210774900292912087961585096411776099985171583447223194199838220638292779801159740619851739001817375494943253704807011777299676604573163468982754146370538969804640016845886088456290907675282514774846922746232994309075209445683945273800365501026275323792787570369388464911734329087864805565520139549955581234034124130958632594892800628123347200151364961638765297220660199483410880838092273881612533044809866049381909541331146300764356286346868367636437251
  public exponent: 65537
  Validity: [From: Tue Nov 14 11:16:07 CST 2017,
               To: Thu Nov 14 11:16:07 CST 2019]
  Issuer: OU=Site Recovery Manager client, O=VMware vSphere Client, C=US
  SerialNumber: [ 01671038 08b4]

]

4. From /var/opt/apache-tomcat/logs/dr.log:

2020-01-09 07:36:50,592 [srm-reactive-thread-12] INFO  com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl$RequestResponseProcessor 30ba2fc4-####-####-####-#########45 pairLogin - Failed trying to retrieve token: ns0:RequestFailed: EndTime: Thu Nov 14 11:16:07 CST 2019 is not after startTime: Thu Jan 09 15:36:50 CST 2020

5. Time synchronization looks good on vCenter/SRM/VR.

6. Restarting vCenter, SRM or vSphere Replication won't help. 

Environment

VMware vSphere Replication 8.x

Cause

The issue is can be caused by expired SRM & VR certificates or services. 

Resolution

NOTE: Take a normal snapshot of the appliance you are renewing the certificate on. 

Try the steps in the order mentioned below and check if it works after each step. 

1. Renew the certificates of SRM or VR, if you find them to be expired 
    
    Change the Site Recovery Manager Appliance Certificate
    Change the SSL Certificate of the vSphere Replication Appliance ​​​​​​​

2. Restart the dr-client & hms service in vSphere replication appliance 

3. Restart the dr-client & srm-server service in Site Recovery Manager appliance