A large number of robots beneath a specific hub appear to be functioning normally (submitting alarms and metrics, probes appear green), but they cannot be directly communicated with. Attempting to retrieve information about the robot in the Admin Console (AC) or Infrastructure Manager (IM) results in an error, probes cannot be opened and configuration/Raw Configure cannot be retrieved.
Symptoms:
Product: DX Unified Infrastructure Management (UIM) - Any Version
hubs configured without hub-to-hub SSL tunnels
The robots may be located behind firewalls and are only configured to communicate with their local secondary hub. The IM client is attempting to communicate with the robots directly, but is being blocked by network firewall policies.
When a hub is connected to another hub (e.g. the primary hub) via a "Name Services" entry, then that hub is assumed to be local (on the same LAN) and additionally all its robots are also considered local.
This means that the IM Client, or the Admin Console app on the primary hub, will try to reach out to the robots directly on their IP address at port 48000.
A firewall restricting access to this port from external sources will block this communication and cause errors.
To resolve this issue, you must configure hub-to-hub SSL tunnels between the secondary hubs and the primary hub to ensure that all management traffic is proxied correctly. When tunnels are present, all communication intended for robots flows through their hubs, and the hub sends the communication to the robot instead of trying to establish a direct path.
Steps to configure tunnels: