UIM Controller port stops responding after working for 10+ minutes

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The UIM controller probe on a secondary hub stops responding to requests.  This includes:

• probe configuration for any probes on this robot
• telnet tests fail (even test to the localhost)

After restarting the robot, all functionality returns.  After some time (10 minutes to as much as 2 hours), the controller again stops responding until it is restarted.

When the issue is occurring, the probe continues to add entries to the log.  There are no errors observed in the controller logs.

Environment

• Windows Server 2016
• UIM 20.4CU5
• hub 9.36
• robot 9.36

Cause

100+ connections to controller port were preventing new connections.

Resolution

• Using "netstat -aon | findstr :48000" at the command line, we observed more than 100 connections to the controller probe from the same, unexpected, IP address. 
• Examining this server, there was a robot installed and reporting to this hub.  The following probes were found on the robot: 
  • ppm
  • baseline_engine
  • prediction_engine. 
• After deleting these probes, the connections were dropped and were not reconnected.  The issues did not re-occur. 
• Other than the unnecessary probes being installed on the other robot, the root cause of these connections being created has not been determined or repeated in testing.