Connection Reset Issue with APIGW 11.1.2 Servers

Products

• CA API Gateway

Issue/Introduction

We have just deployed new apigw servers however we are facing a strange connectivity issue.

Issue:

Connetion to ports 9443 and 8443 fail only when the requests comes from a computer outside the same apigw network.

However the connetion succeds when:

- Accessing through an ssh tunnel

- Connecting directly from another server in the same network as the apigw.

Thorugh ssh tunnel

As a result, we are currently unable to connect properly to the apigw services or the policy manager.

Environment

11.1

Resolution

The Gateway responds correctly when tested from inside the same subnet (curl returns HTTP/1.1 200) or through an SSH tunnel, but connections from outside the subnet fail with a TLS handshake error (Connection reset). This strongly suggests that something along the path (firewall, IPS, load balancer, TLS inspection proxy) is blocking or resetting the session. If it were a Gateway configuration issue (certificates, port binding, ciphers), the problem would occur everywhere, including through the tunnel.

To identify where the reset comes from, one can run the following on the Gateway while reproducing the failed connection:

 

sudo tcpdump -ni <interface> host <CLIENT_IP> and '(port 8443 or port 9443)'

• If the RST comes directly from the Gateway → likely local iptables/nftables rules.

• If the RST comes from another hop → a perimeter firewall or load balancer is closing it.

• If no traffic arrives at all → it is being filtered upstream (ACL, security group, routing).

Immediate checks to perform:

• Verify local firewall rules with nft list ruleset or iptables -S | grep 8443.

• Confirm that ports 8443/9443 are listening on 0.0.0.0 and not just 127.0.0.1 (ss -lntp).

• Check if the perimeter firewall or NGFW is performing TLS inspection and blocking self-signed certificates.

• Ensure ACLs/security groups allow ports 8443/9443, not only SSH (22).

• Confirm that return traffic follows the correct path and no MTU issues are present.

On the Gateway side, we should also confirm in Policy Manager -> Tasks -> Listen Ports that ports 8443/9443 have no client IP restrictions, do not require client certificates, and support TLS 1.2.

A simple test matrix shows:

• External client direct to 8443 → fails

• External client via SSH tunnel → works

• Internal client same subnet → works

• Internal client different subnet (crossing firewall) → likely fails

If the last case also fails, the perimeter firewall is the most probable cause.

Proposed solution:

• On the firewall/NGFW, allow TCP 8443/9443 to 10.xx.xx.xxx and exclude this traffic from TLS inspection (or import the Gateway CA if inspection must remain enabled).

• If the block is local (tcpdump shows reset from the host), update iptables/nftables rules to accept 8443/9443 from the required subnets.

 

So the problem is related to the Network it self, not the GW, the version of Layer7 V11.1.2 and OVAs are always ok so is not a problem that is related to the product,