DR/Vertica backup failing with "rsync: connection unexpectedly closed (0 bytes received so far) [sender]" error

Products

• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

Looking at backup.log file, we can see the backup failing with the following error:

[xxxxx@xxxxxxxx backup_script]$ cat backup.log

Starting backup of database drdata.
Participating nodes: v_drdata_node0001, v_drdata_node0002, v_drdata_node0003.
Snapshotting database.
Snapshot complete.
Approximate bytes to copy: 68051593524 of 170065380325 total.
[=================================================.] 99%Error: Error accessing remote storage: failed accessing remote storage on xx.xx.xx.xx: ssh_exchange_identification: read: Connection reset by peer
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(601) [sender=3.0.7]
: returncode=255

Environment

Release : all supported versions.

Cause

This is most likely related to a timeout hit in ssh connection.

Resolution

Please execute the following procedure:

-----------------------------

1- Just restart the sshd service in destination machine and test if the problem was solved:

systemctl restart sshd

2- If the problem still remains, open the /etc/ssh/sshd_config file and configure the following parameters as below:

TCPKeepAlive yes
ClientAliveInterval 30
ClientAliveCountMax 99999

3- Save the file;

OBS: these parameters are suggested. If they work, but for some reason you need to reduce the values, you can reduce them and test again.

4- Restart sshd service:

systemctl restart sshd

-----------------------------

If the problem still remains after, please contact Broadcom Support Team.