TCP errors seen in the RCPD.OUT file on the Primary SpectroSERVER

Products

• DX NetOps
• CA Spectrum

Issue/Introduction

The following error is seen in the $SPECROOT/SS/RCPD.OUT file on the Primary SpectroSERVER:

Jan 31 13:08:01 : Successfully connected to remote rcpd.  Initiating file transfer... 
Jan 31 13:08:01 : Starting file transfer of size 11596888 bytes using 1048576 byte application buffer, 1048576 byte TCP socket buffer.
Jan 31 13:08:01 SOCKET WRITE WARNING, fd = 4
    Socket backed up, remote application not reading fast enough.
    Retry phase 1 of 4 with 0 second delay for 100 retries.
    Other retry phase messages may follow, but only one SUCCESS or ERROR
    message will result.
    Local Host:  qusnwadq.utcapp.com.2711 (159.82.61.154)
    Remote Host: qusmna5q.utcapp.com.65226 (10.87.248.29)

This is displayed several times and then we see the following:

Jan 31 13:08:06 SOCKET WRITE SUCCESS, fd = 4
    Socket is no longer backed up, remote application has read.
    Problem resolved in:
    Retry phase 2 of 4 with 101 total retries.
    Local Host:  qusnwadq.utcapp.com.2711 (159.82.61.154)
    Remote Host: qusmna5q.utcapp.com.65226 (10.87.248.29)
Jan 31 13:08:06 : /spectrum/SS-DB-Backup/db_20230131_1307.SSdb: has successfully been copied over to qusmna5q
Jan 31 13:08:06 : Waiting for remote rcpd to process the database file...
Jan 31 13:08:32 : The remote rcpd has successfully completed processing.
Jan 31 13:08:32 : Final status is 0

From the RCPD.OUT files on both the primary and secondary, the sync was successful.

Environment

Release : Any
Component: SpectroSERVER

Cause

The error indicates a bottle neck in the TCP buffer between the primary and Secondary SpectroSERVER systems. This could be network or OS related.

Resolution

The following may address the error. If not, the sync is successful and the error can be ignored.

- Log into the Primary SpectroSERVER system

- cd to the $SPECROOT/SS directory

- Create a new file text called .rcpdrc and add the following two lines:

tcp_buffer_size=4194304
send_buffer_size=4194304

- Save the change

- Do the same on the Secondary SpectroSERVER

The next time an Online Backup is run on the Primary SpectroSERVER, when the rcpd process starts, it will read the contents of the .rcpdrc file and set the TCP buffers to the values in the file.

If you still see the error, try increase the values in the .rcpdrc file.