search cancel

tunnel client takes a long time to connect - error message "unexpected status (4) on new connection command"

book

Article ID: 255839

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

We are having an issue where some tunnel clients take a very long time to restart; if you restart the client hub you will see messages like the following repeated for several minutes. 

Nov 10 09:53:12:705 [14888] 0 hub: CTRL <hub name> unexpected status (4) on new connection command 

Eventually it will connect.

Restarting the client hub will make no difference in the length of time it takes to recover, the message will just continue.

Restarting the tunnel server side of the tunnel will generally stop the behavior at the expense of a 3-5 minute down time for all the tunnels connected to the tunnel server. 

How can we avoid this message and allow the tunnels to start up quickly?

Cause

This is related to the max_heartbeat setting at the tunnel server. 

The hub attempts to leave its sessions in a "half-open" state so they can be quickly reused when needed; when a tunnel client restarts, the server-side hub will try to re-use the same SSL/TCP sessions that it was using for that tunnel prior to the restart. 

In many environments, this is not possible because a network device has timed out or closed the session. It has become increasingly common for network switches and firewalls to close or time out half-open sessions; in the past we would recommend that customers disable these timeouts for UIM traffic but many modern network devices no longer allow this and still require some minimum timeout to be enforced, which ultimately leads to difficulties with half-open session usage.

The hub will not time out the sessions on the server side until the max_heartbeat time has expired, and the default for this setting is very high (1800 seconds.)

Resolution

- using Raw Configure, edit the hub configuration
- locate the max_heartbeat key in the <tunnel> section
- set this to 90 instead of 1800

if the key does not exist you may add it directly in the <tunnel> section of the hub configuration.