XCOM Windows to z/OS EBCDIC transfer - apostrophe (') character converted to null

Products

• XCOM Data Transport
• XCOM Data Transport - Windows
• XCOM Data Transport - z/OS

Issue/Introduction

Sending a file from Windows to Mainframe (z/OS) using CODE_FLAG=EBCDIC from Windows.
The apostrophe (') character shows up as null character on the Mainframe.
The Windows file is created from a SQL Server bcp export.

Environment

XCOM™ Data Transport® for Windows 11.6 SP03 GA ("r11.6 20035 SP03")
XCOM™ Data Transport® for z/OS 12.0

Resolution

XCOM Engineering tested with example file C:\tmp\test_file.txt:

test line 1'
test line 2’
test line 3`
test line 4
test line 5

Each line has 12 characters, the last one being a kind of apostrophe or space after the digit.

The hex values are:

PS C:\Program Files\CA\XCOM> od.exe -tx1 -a C:\tmp\test_file.txt
0000000 74 65 73 74 20 6c 69 6e 65 20 31 27 0d 0a 74 65
        t  e  s  t  sp l  i  n  e  sp 1  '  cr nl t  e
0000020 73 74 20 6c 69 6e 65 20 32 92 0d 0a 74 65 73 74
        s  t  sp l  i  n  e  sp 2 dc2 cr nl t  e  s  t
0000040 20 6c 69 6e 65 20 33 60 0d 0a 74 65 73 74 20 6c
        sp l  i  n  e  sp 3  `  cr nl t  e  s  t  sp l
0000060 69 6e 65 20 34 20 0d 0a 74 65 73 74 20 6c 69 6e
        i  n  e  sp 4  sp cr nl t  e  s  t  sp l  i  n
0000100 65 20 35 20
        e  sp 5  sp

The standard apostrophe character (') is 0x27.
XCOM ASCII to EBCDIC (Atoe.tab) translation table is more or less limited to < 0x80.
Therefore it is suspected that the user's file uses an apostrophe character with hex value 0x92 in the Windows file, hence on line 2 it translates to 0x00, non-printable. 

It should be possible to fix that with a UTF transfer Single Byte Character Sets (SBCS). The US code page for 3270 is CCSID#37. Windows US version uses code page windows-1252.

"C:\Program Files\CA\XCOM\XCOMTCP.EXE" -c1 -f queue=NO userid=userid password=password number_of_retries=0 xtrace=0 ckpt=0 compress=NO secure_socket=NO local_file=C:\tmp\test_file.txt remote_file=public.xcom.testfile remote_system=server2 code_flag=UTF8 FILE_OPTION=CREATE port=8044 MAXRECLEN=12 LRECL=12 blksize=12 RECORD_FORMAT=FB REMOTE_CHARSET=CCSID#037 LOCAL_CHARSET=windows-1252 MBCS_CONVERROR=REPLACE

-----------
test line 1'
A8AA498984F7
35230395501D
 -----------
test line 2.
A8AA498984F3
35230395502F
 -----------
test line 3`
A8AA498984F7
352303955039
 -----------
test line 4
A8AA498984F4
352303955040
 -----------
test line 5
A8AA498984F4
352303955050
 -----------

User upgraded from 11.6 SP03 GA to latest 11.6 SP03 PTF LU15408 to resolve another problem and this apostrophe (') translation problem was also resolved at the same time.
However there were no changes to ASCII>EBCDIC translation between the 2 versions and Support also successfully tested the standard apostrophe (') character 0x27 with the GA version in-house.
The data is from a SQL Server export using bcp. Therefore it is suspected there has been some change in that process e.g. perhaps a change in the code page used for the SQL output, which resulted in the use of standard apostrophe (') character 0x27 to resolve this problem.