Secondary hub is not accessible

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

We are unable to access the secondary hub. Getting the below error...But we are able to telnet to the primary and tunnel hubs without any issue. Recycle of robot/hub had no effect, restart no effect. The secondary hub normally connects to the Tunnel Server (Linux).

hub log showed:

Feb 29 00:02:44:445 [4616] hub: nimSession - failed to connect session to 10.xx.xxx.xx:48000, error code 10060
Feb 29 00:02:44:445 [4616] hub: hostname [UIM-xxx-TUN-HUB1-xx] could not connect locally to <nimbus_address> (10.xx.xxx.xx/48000)
Feb 29 00:02:44:445 [4616] hub: hostname sent error message response
Feb 29 00:02:51:460 [3188] hub: nimSession - failed to connect session to xx.xx.xx.xx:48000, error code 10060
...................
Feb 29 00:02:51:460 [3188] hub: hostname sent error message response
Feb 29 00:03:00:103 [2316] hub: Got STOP command from 10.xxxxxxxxxx
Feb 29 00:03:00:103 [2316] hub: main: nimSessionWait rc =4
Feb 29 00:03:00:248 [2388] hub: thread shutdown
Feb 29 00:03:01:234 [2348] hub: CORE exit
Feb 29 00:03:04:116 [2316] hub: EXIT HUB: hub shutdown
Feb 29 00:03:04:116 [2316] hub: Waiting for tunnel to terminate ...
Feb 29 00:03:04:116 [2316] hub: Waited for 0 seconds for tunnel to terminate (OK)

hub was pingable

telnet to the robot port 48000 was successful but not the hub at 48002

But when trying to resolve it, we get non-existent domain and we noted that the other Secondary hubs were 100% resolvable in AWS.

*** awsxxxxxxx.dxxx.com can't find xxxxxxxHUB: Non-existent domain

We checked 3 other Secondary hubs in AWS and they CAN be resolved with no issue.

Environment

Release : Any UIM
Component : UIM - HUB
hub v7.80
Windows 2012 R2

Cause

- Unknown, perhaps something changed in the aws ec2 instance

Resolution

• AWS operations team restored the aws instance from a backup and the hub returned to normal.
• Recommended that the team try to discern what changed, e.g., updates/changes, changes to configuration, as per the aws System logs.
• hub came back online 100% after restoring from backup in AWS.