SRM VR and SaaS showing not connected

Products

• VMware Live Recovery

Issue/Introduction

To improve overall vSphere performance.  Remove RPOs and allow easier connections between sites.

• Cloud SRM connection shows Not connected
• SRM connection shows Not connected
• VR  connection shows Not connected
• Replications show RPO.
• Replications stuck at 99%
• Logs on appliances show consistent resets.

Environment

VMware vSphere Replication 8.x

Cause

• High IO environments, 
• Edge deployments.  
• Unavoidable network performance pitfalls. 

Resolution

Ensuring Connectivity Between Sites

1. Obtain the FQDN of the DRaaS Appliances
   To find the Fully Qualified Domain Name (FQDN) of the DRaaS appliances, go to the vmc.xx.com console for specific SDDC. Navigate to the "Add Ons" tab, where the SRM FQDN for each instance will be displayed. The "OPEN SITE RECOVERY" link leads to the vSphere Replication FQDN.

2. Testing Connectivity with curl Commands
   Run the following curl commands from the on-prem appliances (vCenter, SRM, and VRMS) to check DNS, routing, firewall, and connectivity. Each command should show a successful connection, display remote certificate information, and return output resembling the source of a small web page.

    

   curl --verbose https://vcenter.sddc-##-##-##-##.vmwarevmc.com/

   curl --verbose https://vr.sddc-##-##-##-##.vmwarevmc.com/

   curl --verbose https://srm.sddc-##-##-##-##.vmwarevmc.com/

   Key Ports for Connectivity:

   • VR: 443, 8123
   • SRM: 443, 9086
   • vCenter: 443, 80

 

RPOs and Connection Resets

To ensure optimal replication, the best network configuration allows both 1500 and 9000 non-fragmented pings between networks.

• MTU (Maximum Transmission Unit) defines the maximum size of a data frame that can be transmitted over the network without fragmentation.

We have tools to address potential issues:

1. Enable MTU Discovery on VR/SRM Appliances
   Run the following command on the VR and SRM appliances to enable MTU discovery when an ICMP black hole is detected:

   sysctl -w net.ipv4.tcp_mtu_probing=1
   

   This command ensures that the MSS (Maximum Segment Size) is calculated based on the actual path MTU, preventing packets from exceeding the MTU and requiring fragmentation

2. Replication Job Considerations
   After enabling MTU discovery, recreate the replication job from the beginning with network compression enabled to optimize performance.

3. Increasing TCP Retries for Site Pairing/Connection
   For site pairing or connections, increase the TCP retries by running the following command:

   echo 'net.ipv4.tcp_syn_retries=3' >> /etc/sysctl.d/90-va-tune-up.conf

Additional Information

Impact/Risks:
Low