Large file transfer performance with XCOM for Windows

Products

XCOM Data Transport XCOM Data Transport - Windows

Issue/Introduction

The requirement is to transfer a 400GB backup file at a given point in time between 2 Windows servers.
Unfortunately with the default configuration on both ends it took about 165 minutes to transfer one 2.6GB compressed file with no other active transfers.
Is there any configuration tuning that can be done to improve the File transfer rate?

Environment

XCOM™ Data Transport® for Windows

Resolution

Removing the "XTRACE=10" command line parameter reduced transfer time from 165 minutes to 5 minutes.
At the same time, CACHE_READ_SZ and CACHE_WRITE_SZ were changed to 5120. However they are really helpful only when the disk I/O is slow compared to Network speed. Unless there are Disk I/O issues, they probably won't help much unless a specific test with those changes shows an improvement:
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Using > How to Provide Performance Tuning
Further suggestion: The COMPRESS option (default is YES for 11.6, NO for 12.0)
XCOM Data Transport FOR WINDOWS 12.0 COMPRESS
a. Run a test with COMPRESS=NO in the .cnf/command-line parameters and see if there is any further improvement on the 5 minutes result.
b. Run a test with COMPRESS=ZLIB2 in the .cnf/command-line parameters and again compare with the 2a result and the "COMPRESS=YES" 5 minutes result.
Additional:
a. If the transfer is Binary (CODE_FLAG=BINARY), set the .cnf/command-line parameter MAXRECLEN to a higher value.
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > How to Configure CA XCOM Data Transport > XCOM.CNF Parameters > MAXRECLEN
b. If the network is reliable and restarts are not expected, set the .cnf/command-line parameter Checkpoint (CKPT/CHECKPOINT_COUNT) to zero. To play safe, use a higher CKPT value.
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > Reference > List of Parameters > CKPT
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > How to Configure CA XCOM Data Transport > XCOM.CNF Parameters > CHECKPOINT_COUNT
c. Use a higher value for xcom.glb parameter STAT_FREQUENCY. It can cause the display of slightly older values when monitoring the transfer progress when it is active.
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > How to Configure CA XCOM Data Transport > XCOM.GLB Parameters > STAT_FREQUENCY
Starting with SP03, an additional CARRIAGE_FLAG parameter called ZPACK, was added allowing users to configure/specify up to 64k TCP/IP buffers by using the MAXRECLEN and XBUFFSIZE parameters. In house tests have shown a 25% improvement in transfer rates of very large gigabit files, BUT both systems will require to be at a SP03 level.
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > How to Configure CA XCOM Data Transport > XCOM.CNF Parameters > CARRIAGE_FLAG
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > How to Configure CA XCOM Data Transport > XCOM.GLB Parameters > XBUFFSIZE
A suggested extra test if transfer is binary ("CODE_FLAG=BINARY") and both systems are not at SP03.
Using whatever above "COMPRESS" value is found to best:
.cnf/command-line parameters:
CARRIAGE_FLAG=XPACK
NOTE: CARRIAGE_FLAG=YES is the default and per above doc ref for TCP/IP transfers it should be automatically changed to XPACK unless MAXRECLEN >=31744. Set XPACK explicitly to be sure.
MAXRECLEN=31744 (the default is 1024)
xcom.glb parameters:
STAT_FREQUENCY=1000 (the default is 10)
XCOM DATA TRANSPORT FOR WINDOWS 11.6 SERVICE PACKS > Administrating > How to Configure CA XCOM Data Transport > XCOM.GLB Parameters > STAT_FREQUENCY

Additional Information

Although most of the links are for release 11.6, the information will apply to release 12.0.
When clicking on the links, you may select the 12.0 documentation from the drop down link.