wasp probe failed to start - cannot obttain connection string from data_engine / Database information is null / Max. restarts reached

Products

DX Unified Infrastructure Management (Nimsoft / UIM) CA Unified Infrastructure Management On-Premise (Nimsoft / UIM) CA Unified Infrastructure Management SaaS (Nimsoft / UIM)

Issue/Introduction

This article provides a solution to an issue where the wasp probe is not starting, and ends up in a failed (red) state when it fails to locate the data_engine probe or throws a Max. restarts error and does not start up successfully.

The wasp.log, and/or _wasp.log files will show similar errors to the below snippet:

[main, com.nimsoft.nimbus.NimProbeBase] Login to NimBUS is OK
[main, com.nimsoft.nimbus.probe.service.wasp.WaspNimProbe] WaspNimProbe login successful.
[main, com.nimsoft.nimbus.probe.service.wasp.Probe] Connection String: null
[main, com.nimsoft.nimbus.probe.service.wasp.Probe] Cannot obtain connection string from data_engine
[main, com.nimsoft.nimbus.probe.service.wasp.Probe] (4) not found, Received status (4) on response (for sendRcv) for cmd = 'nametoip' name = 'data_engine'
 at com.nimsoft.nimbus.NimSessionBase.sendRcv(NimSessionBase.java:610)
 at com.nimsoft.nimbus.NimSessionBase.sendRcv(NimSessionBase.java:561)

The wasp probe will be in an error state (no PORT, no PID and in red status)

or you will see errors such as:

ERROR [main, com.nimsoft.nimbus.probe.service.wasp.Probe] main() Fatal error! 
ERROR [main, com.nimsoft.nimbus.probe.service.wasp.Probe] main() (1) error, Database Information is null 
ERROR [main, com.nimsoft.nimbus.probe.service.wasp.Probe] (1) error, Database Information is null 
    at com.nimsoft.nimbus.probe.service.wasp.Probe.doit(Probe.java:327) 
    at com.nimsoft.nimbus.probe.service.wasp.Probe.main(Probe.java:119) 
[5976] Controller: Max. restarts reached for probe 'wasp' (command = <startup java>)

or

INFO  [main, com.nimsoft.nimbus.NimProbeBase] Login to NimBUS is OK
INFO  [main, com.nimsoft.nimbus.probe.service.wasp.WaspNimProbe] WaspNimProbe login successful.
INFO  [main, com.nimsoft.nimbus.probe.service.wasp.Probe] Connection String: 
DEBUG [main, com.nimsoft.nimbus.lookup.DataEngineLookup] Request to probe "/<domain>/<hub>/<robot>/data_engine" callback get_connection_string was successful.
DEBUG [main, com.nimsoft.nimbus.utils.FileUtils] In FileUtils.getFileChecksum
DEBUG [main, com.nimsoft.nimbus.utils.FileUtils] local certificate store FileChecksum - 561c2ed21da6f084266d3c3b67950e09
[7096] Controller: Max. restarts reached for probe 'wasp' (command = <startup java>)

Environment

Release: DX UIM 20.x or 24.x
wasp probe

Cause

Upon review of the wasp.cfg, either the probe is pointing to the following location to locate the probe:

data_engine = data_engine

or there is no data_engine parameter and value defined in the wasp.cfg.

This is incorrect. The wasp is usually deployed to a separate robot which attaches to and is under the Primary hub system and in this use case the key should contain the full NimBUS address of the data_engine, for example here is the syntax:

data_engine = /<uim_domain>/<primary_hub_name>/<primary_hub_robot_name>/data_engine

Resolution

1. Go to the Primary hub server, right click on the data_engine probe, go to the bottom line for "Edit" and select "Copy to Clipboard"

2. Open a notepad file and paste the contents. Locate the data_engine probe string.

e.g., "/<uim_domain>/<primary_hub_name>/<primary_hub_robot_name>/data_engine"

3. Copy the probe connection string

4. Open the wasp probe in Raw Configure mode, and under the setup section, locate the data_engine key and edit the key

5. Enter the data_engine NimBUS address:

e.g., "data_engine = /<domain>/<hub>/<robot>/data_engine"

Note: NimBUS addresses are case-sensitive! Make sure to use the proper case for the entire address to match your actual domain and robot addresses.

6. Click Apply to apply the changes and Save.

7. Restart the robot service of the OC robot system/Deactivate-Activate the wasp probe.

8. Check the wasp log at loglevel 5 to see if the issue is resolved.

Additional Information

If the wasp probe still fails to start:

Check the presence and size of the certificate.pem file

Ensure that you have copied the exact same certificate.pem file from the Primary hub and pasted the file into a directory on the robot where the wasp is deployed.

Do NOT try to copy the contents of the file into another file named certificate.pem, otherwise it may end up as a slightly larger or smaller file, and even if it is off by a single byte, the wasp will not start.

Update the robot.cfg

Reference the file using the correct path in the robot.cfg, and click Apply, for example:

cryptkey = C:\Program Files (x86)\Nimsoft\robot\security\certificate.pem

What is the certificate.pem file?
The DX UIM Server installer creates a .pem file (certificate.pem) in the <Nimsoft>\security folder. The .pem file is a symmetric key that is shared with the required robots, which is then used for communication with the data_engine probe. You copy this .pem file to the remote Operator Console and CABI robots, and provide the location of the file in the robot.cfg file (cryptkey = <.pem file location>).

Furthermore, when the certificate.pem is refererenced in the robot.cfg, make sure there are no single or double quotes around the entry, otherwise it will not be able to be read.
Check the wasp probe java memory settings

Check the java heap memory settings under the wasp startup options section of the wasp.cfg.

Make sure it has enough memory configured and/or that the memory configured doesn’t surpass the RAM that is available on the system.

Normally the minimum memory should be set to a minimum of 1024 and max of 2048, or 2048 and 4096, but if the memory requirements are set higher for some reason, e.g., wasp on the cabi robot, or someone increased the RAM beyond the physical memory that is available, the wasp may not startup successfully and it throws a Max. restarts error.