We have changed their networks around. After changing the IP addresses in the appropriate configuration files and verifying that Firewall NATs were in place to allow correct ports to communicate between the tunnel client hub and the tunnel server proxy hub, the tunnel client cannot establish a tunnel connection.
We have deleted and recreated the tunnel cert and reapplied it on the tunnel client hub. But still cannot get it to connect.
The hub.log file from the tunnel client shows the client is trying to establish a tunnel connection to the tunnel server (on configured tunnel server NAT IP address ##.##.#.#):
Jul 27 11:51:40:372 [140353272530688] hub: SSL handshake start from ##.##.#.#/48003: before/connect initialization
Jul 27 11:51:40:372 [140353272530688] hub: SSL state (connect): before/connect initialization
Jul 27 11:51:40:372 [140353272530688] hub: SSL state (connect): SSLv3 write client hello A
But, the SSL connection is failing with an SSL_accept error (5) on the tunnel server side (from the configured tunnel client NAT address ##.##.#.#):
Jul 27 11:51:46:451 [3948] hub: SSL handshake start from ##.##.#.#/57294: before/accept initialization
Jul 27 11:51:46:451 [3948] hub: SSL state (accept): before/accept initialization
Jul 27 11:51:46:451 [3948] hub: ssl_server_wait - SSL_accept error (5) on new SSL connection: ##.##.#.#
Usually, when we see SSL error 5 in the hub logs it is because there is a security device on one of the networks that is doing a deep inspection or changing the SSL traffic in some way.
Basically, the hub is saying the SSL packet is not valid and it has been tampered with.
These errors a very hard to track down.
They require engaging the network teams on both sides to analyze the traffic and see what is tampering with the packets and then putting a rule in place to prevent it.
This behavior is very characteristic of firewall interference and does not appear to be anything related to the configuration of the hubs themselves.
Something in between these two hubs appears to be blocking the SSL handshake.