One or more robots has registered itself with the IP address 127.0.0.1, or 169.x.x.x which has prevented the robot from properly submitting data and alerts to the hub.
Example in log:
Aug 30 04:38:55:441 [3240] Controller: ----- Robot controller ********************** started -----
Aug 30 04:38:55:441 [3240] Controller: Search for IP address failed - not found
Aug 30 04:38:55:441 [3240] Controller: Warning - Robot IP is set to loopback, communications with remote hubs will be impossible
Can this be prevented?
Kwon issue. Rebooting servers would sometimes bring up the robot using the loopback address (127.0.0.1) on the server. In some cases, this seems to be caused by a 'race condition' between UIM starting and the network going active. So it may also still be worth setting the Nimsoft Robot Watcher service to 'Automatic (Delayed Start),' on Windows if the upgrade and parameters listed below do not avoid the issue of the robot taking the loopback or APIPA address.
Additionally, it is common when adding a virtual adapter in VMware running Windows in that the adapter will initially go live with a default IP configuration which includes DHCP. It is also common to leave the adapter disconnected from a virtual switch until configured which will prevent it from reaching a DHCP server. If the server or UIM is restarted, then on startup it seems to be fairly likely that the controller will bind to the 169.x.x.x DHCP failure address in this case.
Please refer to to robot (controller) release notes:
controller Release Notes
Two configurable parameters to be added in the robot.cfg are available and have proven to alleviate or resolve the problem.
max_retry_IfLoopBack_attempt
Specifies the maximum number of retries that controller can use to find the system IP address. In the case of loopback IP address, if robot_ip is not configured in robot.cfg, then controller tries to get the system IP. It tries to find the IP address for the number of retries that are specified in the parameter. After trying for the specified count, controller starts again with the loopback IP, resulting into the same initial behavior. The default value is 10.
sleep_btwRetry_IfLoopBack_attempt
Specifies the maximum interval for which controller waits before trying again for the subsequent attempt. The default value is 2 seconds.
Several UIM admins indicated that this appears to occur after the robot IP address is changed and the system is rebooted. Note that other UIM customers have noticed this behavior after a reboot as well.
In robot_update 9.20 to 9.34, these parameters will cause the robot to engage in the above behavior when its IP address is 127.0.0.1.
In robot_update 9.35 and higher, these parameters also detect the issue with 169.x addresses.
To further ensure that the robot will no longer use the loopback/APIPA address after an IP change and subsequent reboot, in the robot.cfg controller section, you can try the following parameters:
max_retry_IfLoopBack_attempt = 10
sleep_btwRetry_IfLoopBack_attempt = 5
If you expect outages to be significantly longer you can set the number of retries higher, and sleep time higher to accommodate longer periods.