Unable to reach controller on linux robots but QOS works

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)
• CA Unified Infrastructure Management On-Premise (Nimsoft / UIM)
• CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

We have issues on 2 Linux servers, where all installed probes are giving the same error and error is "Unable to reach controller"

Full Error: 
"Unable to reach controller, node: /uim/primary_hub/hub/cdm, error message: communication error."

 

However, QOS is flowing correctly from the robots to the hub. 

 

•  There is no firewall between UIM and linux servers, local firewall is disabled on both sides.

•  I can telnet port 48000 on both sides.    

•  I have tried to reinstall robots, then delete and fresh install.

•  I have deleted cache. 

•  Robot version is 9.35, all probes are green in Infrastructure Manager. 

Environment

Release : UIM 20.x - Robot 9.x

Cause

The problematic machines might be taking more time for controller "get_info" callback to execute, so the reason for the unreachable controller boils down to the slow networking stack. As it is taking more time to resolve the hostname, we see this issue. The get_info command that is taking 40 seconds to respond, has the functionality which interacts with the underlying OS to get network-related information.

From the logs, it is visible that the area where the system calls "gethostname" and "gethostbyname" is used, is taking 40 seconds of time.

These are the OS system calls which interact with the underlying networking stack to resolve the local hostname, if the underlying networking stack is slow then these callbacks will take more time.

This explains the controller GUI not be reachable while the robot actually works (QOS Flow is correct) 

 

 

 

Resolution

It is suggested to check how hostnames are resolved, if DNS is used for hostname resolutions then check the responsive speed of the DNS server.