Nimsoft robot goes down intermediately.

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

We have observed in many servers the Nimsoft Service Controller status information does not show and all the operational buttons gets disabled. Due to this we get false alarms in nas.

Environment

Version: Any
Component: robot

Cause

Looking at controller.log, as we can see for every few minutes the Robot is losing communication with Hub and it failed to connect on hub ports... 

Jun 6 15:23:13:812 [10820] Controller: hub <hubname><ipaddress> NO CONTACT (communication error) 
Jun 6 15:24:14:192 [10820] Controller: hub <hubname><ipaddress> NO CONTACT (communication error) 
Jun 6 15:25:01:196 [10820] Controller: async_req - nimSessionAConnect failed for <ipaddress>/48001 (error)
Jun 6 15:25:01:196 [10820] Controller: async_req - nimSessionAConnect failed for <ipaddress>/48007 (error) 

Due to this communication problem, the Nimsoft Service Controller status fluctuates, also telnet from Robot to Hub fails for every few minutes and then it connects.

Another issue is Windows port exhaustion. If you see hundreds of these in the 65000 range, this will be a symptom of port exhaustion.

netstat -an

  TCP    <ip address>:61882   <ip address>:1433      ESTABLISHED
  TCP    <ip address>:61886   <ip address>:1433      ESTABLISHED
  TCP    <ip address>:61888   <ip address>:1433      ESTABLISHED
  TCP    <ip address>:63091   <ip address>:48002    ESTABLISHED
  TCP    <ip address>:63516   <ip address>:48002    ESTABLISHED
  TCP    <ip address>:63517    <ip address>:48002   ESTABLISHED
  TCP    <ip address>:63518    <ip address>:48002   ESTABLISHED
  TCP    <ip address>:65080    <ip address>:1433     ESTABLISHED

Resolution

From the above symptom as telnet fails to connect to Hub, this appears to be an Environment problem.

1) We could disable Anti-Virus application for few minutes and check telnet works without any disruption
2) Run wire-shark trace to see the connectivity issues between Robot to Hub
3) Reboot the robot machine due to Port Exhaustion