XCOM passwords and special character support

Products

• XCOM Data Transport
• XCOM Data Transport - Linux PC
• XCOM Data Transport - Windows
• XCOM Data Transport - z/OS

Issue/Introduction

Using XCOM for Windows 11.6 sp03.
As part of security compliance from cyber team, we are in process to renew the passwords to transfer the files.
Are there any special characters which XCOM does not support the use of in passwords ?
For example:
*
& 

Environment

XCOM™ Data Transport® for Windows
XCOM™ Data Transport® for UNIX/Linux
XCOM™ Data Transport® for z/OS

Resolution

Table of Contents

• XCOM Data Transport
• Operating System

XCOM Data Transport

XCOM has internal tables that perform ASCII to EBDCIC and EBCDIC to ASCII conversions when transfers are sent.
Those tables were created 40 years ago from IBM SYSTEM Reference Summary Code Translation Tables.
The tables are set up for a US environment, that is typically Windows-1252 or CP437 on the Windows ASCII side and code page 037 on the z/OS EBCDIC side.
Therefore XCOM should support any ASCII character with decimal code < 128 (0x80).

The tables may work for the basic characters in a French, German, Italian etc. environment, but there can be problems with special characters. Countries other than the US could be the first cause of problems.
Even if transfer from an ASCII to an ASCII environment, for example Windows to Linux, the data which describes the transfer, userid, password, file names and so on,  will be translated from the ASCII environment to the XCOM standard environment which is considered to be EBCDIC and back to ASCII on the receiving side.
This could be the second cause for problems, in particular, if the ASCII <> ASCII environment happens to be in France, Germany etc.

There is however a way to verify if the password characters are supported
Copy the password into a text file, send this text file with the parameter CODE_FLAG=EBCDIC using a loopback transfer to the XCOM server itself. The data in the transferred file will have undergone the same treatment as the password.
For example, attached file AllByteHexValues_0x00_to_0xFF.bin has all binary values from 0 to 255.
Send it to file AllByteHexValues_0x00_to_0xFF_received.bin using CODE_FLAG=EBCDIC and CARRIAGE_FLAG=NO.
With this CODE_FLAG=EBCDIC, XCOM will translate all binary values 0-255 ASCII>EBCDIC and back.
Perform a hex dump of the binary values in the original file and transferred file to get attached file AllByteHexValues_0x00_to_0xFF_od.txt and AllByteHexValues_0x00_to_0xFF_received_od.txt respectively.
Comparing the 2 files there is a match for values < 128 (0x80) and anything above ends up 0.

All values 0-127 match:

000000 nul soh stx etx eot enq ack bel bs  ht  nl  vt  ff  cr  so  si
       00  01  02  03  04  05  06  07  08  09  0a  0b  0c  0d  0e  0f
000010 dle dc1 dc2 dc3 dc4 nak syn etb can em  sub esc fs  gs  rs  us
       10  11  12  13  14  15  16  17  18  19  1a  1b  1c  1d  1e  1f
000020  sp   !   "   #   $   %   &   '   (   )   *   +   ,   -   .   /
       20  21  22  23  24  25  26  27  28  29  2a  2b  2c  2d  2e  2f
000030   0   1   2   3   4   5   6   7   8   9   :   ;   <   =   >   ?
       30  31  32  33  34  35  36  37  38  39  3a  3b  3c  3d  3e  3f
000040   @   A   B   C   D   E   F   G   H   I   J   K   L   M   N   O
       40  41  42  43  44  45  46  47  48  49  4a  4b  4c  4d  4e  4f
000050   P   Q   R   S   T   U   V   W   X   Y   Z   [   \   ]   ^   _
       50  51  52  53  54  55  56  57  58  59  5a  5b  5c  5d  5e  5f
000060   `   a   b   c   d   e   f   g   h   i   j   k   l   m   n   o
       60  61  62  63  64  65  66  67  68  69  6a  6b  6c  6d  6e  6f
000070   p   q   r   s   t   u   v   w   x   y   z   {   |   }   ~ del
       70  71  72  73  74  75  76  77  78  79  7a  7b  7c  7d  7e  7f

No match for values above 127:

000080 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
       00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
0000a0 nul nul nul nul nul nul   ~ nul nul nul nul nul nul nul nul nul
       00  00  00  00  00  00  7e  00  00  00  00  00  00  00  00  00
0000b0 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
       00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
0000c0 nul  rs  fs nul nul nul nul nul nul nul nul nul nul nul nul nul
       00  1e  1c  00  00  00  00  00  00  00  00  00  00  00  00  00
0000d0 nul   # nul nul nul nul nul nul nul nul nul nul nul nul nul nul
       00  23  00  00  00  00  00  00  00  00  00  00  00  00  00  00
0000e0 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
       00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00
0000f0 nul   q nul nul nul nul nul nul nul nul nul nul nul nul nul nul
       00  f1  00  00  00  00  00  00  00  00  00  00  00  00  00  00

The binary codes of the characters which do not match are all above 127.

 

Operating System

Some Operating System platforms including the security package being used may limit certain characters.
This has been reported outside of XCOM after moving from on-premise to Amazon AWS EC2 Cloud environment e.g. password characters {, |, }, (0x7C, 0x7D, 0x7E) which were previously used successfully for Windows users on-premise, were not accepted under AWS Cloud EC2. Those 3 characters and a few others as often being restricted on Windows for system/security or other reasons.

Additional Information

Web reference for standard US Windows code page 1252: ASCII Table > Reference to ASCII Table of Windows-1252

XCOM for Windows Passphrase support to allow passwords of up to 100 characters was introduced in a September 2018 11.6 SP02 PTF SO05626 and is available in the 11.6 SP03 version (released in March 2020).
That enhancement is covered in this doc. section: XCOM™ Data Transport® for Windows 11.6 Service Packs > Release Notes > New Features