HCX DR Recovery taking over 4 hours or fail even for small VMs due to one of the site pairing in disconnected state

Products

• VMware HCX

Issue/Introduction

• HCX Site Pairing changing status intermittently as Pending or Connected the Connector side, while the Cloud side may show as Connected or Disconnected.
• There is one (or more) site pair that is continuously in disconnected state (may be for a known reason).
• Editing and re-entering site pairing credentials does not resolve the issue.
• VM Migrations using HCX DR Recover performance is drastically reduces, may take 4 hours for a small VM or even fail.
• SyncNow operations for DR-protected VMs fail to complete or remain in a queued state.
• In HCX Manager /common/logs/app.log log file may be filing up with errors pointing to the IP address of the site pair that is continuously in a disconnected state:

  [EventingService_SvcThread-671945, Ent: HybridityAdmin, , TxId: ########-####-####-####-############] ERROR c.v.v.h.s.e.c.EventingRestClient- EventRemoting error, remote: ################-########-####-####-####-############, response: {
      "status": "FAILURE",
      "failure": "ConnectTimeoutException",
      "details": "org.apache.http.conn.ConnectTimeoutException: Connect to <SitePairIP>:443 [/<SitePairIP>] failed: connect timed out\n\tat 

Resolution

This is a known issue impacting VMware HCX. Subscribe to this KB article to be updated if a fix version becomes available. 

If you believe you have encountered this issue, open a support case with Broadcom Support and refer to this KB article.

For more information, see Creating and managing Broadcom support cases.