Deployment of probes to secure robots via automated_deployment_engine fails

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

Deployment of any probe to secure robot fails using automated_deployment_engine probe with below error message seen on admin console deployment activity

 

Distribution of probes to same secure robot works via distsrv probe

 

 

Environment

Release : 20.3.x

Component : UIM - ROBOT

automated_deployment_engine 20.31 
distsrv 9.33
controller 9.32 (secure)

Resolution

This is not a ADE specific issue. This is about availability of threads. In secure bus , controller proxy deals with every communications a probe does outside. For that purpose it has been multi threaded.

By default according to the system configuration the threads for the incoming and outgoing connections on the robot will be selected by the proxy controller. In scenario where system specs are not high , so the default allocated threads are not sufficient.

Below parameters are used to increase the threads externally.

proxy_wq_inbound_max_threads 
proxy_wq_outbound_max_threads 

 

 

This setting may not be needed in all robots. This issue is usually observed only on the robots where OC or CABI  is installed. 
However if the issue is observed on another secure robots these parameters can still be used .

 
During the startup, these threads will be allocated depending upon the underlying server resource configuration. But on a busy robot the default created threads may not be sufficient if the underlying configuration is low.

For this purpose the parameters has been externalized so that they can be increased on need basis.

 For example in this case it has allocated only one inbound and one outbound thread, as the configuration seems to be too low. 

So, the robot was not able to communicate outside correctly. When the value was increased it started working.

Can search for "proxy setup: proxy wq is set to inbound threads"

controller loglevel 6

 “Apr 22 07:07:30:855 [8720] 2 Controller: proxy setup: proxy wq is set to inbound threads 1 and out bound thread 1”

Example settings

Can check to bring these values between 5-10 on hub controller

Can check the hub controller log statement for how many threads are allocated  .Can add below in hub robot.cfg if lower values are observed 

proxy_wq_inbound_max_threads = 10
proxy_wq_outbound_max_threads = 10

 

If other secure robots also have one thread for inbound and outbound then suggested to have them with value around 5

proxy_wq_inbound_max_threads = 5
proxy_wq_outbound_max_threads = 5

Note:This needs nimsoft service restart on concerned hub and robot i.e where the settings are added / changed