XCOM using AT-TLS: XCOMM0780E "Txpi 215: Socket send error return value = 140"

Products

XCOM Data Transport - z/OS XCOM Data Transport XCOM Data Transport - Linux PC XCOM Data Transport - Windows

Issue/Introduction

Configured Port 8045 as AT-TLS Secure port to connect to the mainframe z/OS server. Incoming transfers are failing with the following messages:

XCOMM0780E IPv4 Txpi 215: Socket send error return value = 140
XCOMM0780E IPv4 Txpi 228: Select exception condition for socket 0

A second reported occurrence for a transfer between 2 LPARS in same Sysplex where AT-TLS had been configured for both LPARs in the same way with AT-TLS_PORTS = 8048. A TLS trace showed:

EZD1287I TTLS Error RC: 5003 Data Decryption 312
LOCAL: ##.##.#.#..8048
REMOTE: ##.##.#.#..57729
JOBNAME: XCOM RULE: XCOM_Client_Rule
USERID: ##### GRPID: 00000036 ENVID: 00000000 CONNID: 0015B16C

Job was also showing EZD1286I RC 202 and RC 5006 plus ACF2 USS violation report showed:

R_datalib ###### HT342 0 0
09/01/25 25.244 20.31.30 TCPIP LPAR1 LPAR1
Failed - Profile for Ring_name not found

R_datalib ###### HT342 0 0
09/01/25 25.244 20.31.30 TCPIP LPAR1 LPAR1
Failed - dbToken error.

Environment

XCOM™ Data Transport® for z/OS 12.0

Cause

SSL certificate related.

Resolution

1. In the first reported occurrence the certificates had expired. To create new SSL certificates contact Security team and when using SSL certificates ensure site specific procedures are followed.

2. In the second reported occurrence each certificate was valid but had been created for the host name including domain (FQDN) but not for the IP address being used in the XCOM transfer job IPNAME parameter. The certificate was updated to also have the IP address included which resolved the problem.

Additional Information

Troubleshooting

From IBM z/OS 3.2 > Return codes (errnos), return code 140 means:

140 008C EPIPE The pipe is broken.

AT-TLS reports 140 in case of SSL handshake failure.

The following steps can help:

Check the XCOM partner side log file.
Run a z/OS side AT-TLS trace with IBM's assistance.
The doc. page IBM z/OS 3.2 > Steps for diagnosing AT-TLS problems provides steps for debugging AT-TLS which can be used while waiting for IBM to be engaged to run a trace.
Note: For USS command "pasearch -t" to work, requires enough authority to run the command, otherwise error "EZZ8435I pasearch Command: Environment Error 21" may occur.
In the report look for local and remote port combination problems.
Finding AT-TLS error messages depends on how the TTLS rules have been configured.
For example, for an odd trace value messages will be found in the TCPIP address space. See trace in IBM z/OS 3.2 > TTLSEnvironmentAction statement
To simulate problems z/OS AT-TLS client intercepted traffic was directed to a Windows non-SSL port (8044) and with an odd value for the trace EZD messages are seen in the TCPIP address space e.g.
TYPE=EXECUTE job log:
*****
SDSF OUTPUT DISPLAY XCOMAWCL JOB96754 DSID 2 LINE 23 COLUMNS 02- 133
COMMAND INPUT ===> SCROLL ===>CSR
16.24.07 JOB96754 +XCOMM0785I remote_ip #002000 WINSND01 STARTING TCP/IP CONNECTION TO PORT=8044, IP=remote_ip16.24.07 JOB96754 +XCOMM0786I remote_ip #002000 WINSND01 TCP/IP CONNECTION ESTABLISHED WITH DEST=**NONE**, PORT=8044, IP=remote_ip
16.24.07 JOB96754 IGD103I SMS ALLOCATED TO DDNAME SYS00006
16.24.07 JOB96754 +XCOMM0402I remote_ip #002000 WINSND01 REQUEST NUMBER 002000 ASSIGNED TO TRANSFER REQUEST
16.24.07 JOB96754 +XCOMM0780E remote_ip #002000 WINSND01 Txpi 215: Socket send error return value = 140
16.24.07 JOB96754 +XCOMM0783E remote_ip #002000 WINSND01 FLUSH REQUEST ENDED DUE TO TCP/IP ERROR
16.24.07 JOB96754 IGD104I USER1234.DEMO.ZOSSRC.LRECL80.FB.S000002 RETAINED, DDNAME=SYS00006
*****
TCPIP address space job log showing EZD messages:
*****
SDSF OUTPUT DISPLAY TCPIP31 STC73263 DSID 2 LINE 262 COLUMNS 02- 133
COMMAND INPUT ===> SCROLL ===>CSR 16.24.07 STC73263 EZD1287I TTLS Error RC: 5003 Data Decryption 062
062 LOCAL: local_ip..52415
062 REMOTE: remote_ip..8044
062 JOBNAME: XCOMAWCL RULE: XCOM-XCOMAW-Client~36
062 USERID: USER1234 GRPID: 00000001 ENVID: 000004DC CONNID: 0038ADFA
16.24.07 STC73263 EZD1287I TTLS Error RC: 406 Initial Handshake 063
063 LOCAL: local_ip..52415
063 REMOTE: remote_ip..8044
063 JOBNAME: XCOMAWCL RULE: XCOM-XCOMAW-Client~36
063 USERID: USER1234 GRPID: 00000001 ENVID: 000004DC CONNID: 0038ADFA
*****
The TTLS codes are explained here: IBM z/OS 3.2 > AT-TLS return codes
If the partner is Linux or Windows.
As the XCOM SSL ports 8045 and 8047 may both in use, this test uses the next free port 8048.
Start a listener using port 8048 with the default XCOM SSL root and server certificates & key:
openssl s_server -port 8048 -cert /opt/CA/XCOM/ssl/certs/servercert.pem -CAfile /opt/CA/XCOM/ssl/certs/cassl.pem -key /opt/CA/XCOM/ssl/private/serverkey.pem
At this point start an XCOM z/OS AT-TLS transfer to port 8048 to see the details of the SSL handshake.
To also demonstrate the client side, use a Windows machine to connect to the above port 8048 (or as the case may be to the XCOM z/OS AT-TLS port):
"C:\Program Files\CA\XCOM\openssl.exe" s_client -host 1.2.3.4 -port 8048 -cert "C:\Program Files\CA\XCOM\ssl\certs\clientcert.pem" -CAfile "C:\Program Files\CA\XCOM\ssl\certs\cassl.pem" -key "C:\Program Files\CA\XCOM\ssl\private\clientkey.pem"

Documentation/articles

Related article: XCOM configuration for AT-TLS