nfa_inventory "Failed to execute origin mapping, empty results" after Primary hub service is restarted

Products

• NIMSOFT PROBES
• DX Infrastructure Management

Issue/Introduction

NFA Integration is working but the nfa_inventory log captures the error below:

[interfaceToOriginMapping, nfa_inventory] Executing batch query from #### to #####.
[interfaceToOriginMapping, nfa_inventory] Received unexpected failure executing origin mapping service. (11) command not found, Received status (11) on response (for sendRcv) for cmd = 'dispatcher'
[interfaceToOriginMapping, nfa_inventory] Failed to execute origin mapping service, returning empty results.
[interfaceToOriginMapping, nfa_inventory] java.lang.RuntimeException: (11) command not found, Received status (11) on response (for sendRcv) for cmd = 'dispatcher'
 at com.ca.shift.client.RequestPoolRemoteEndpoint.invoke(RequestPoolRemoteEndpoint.java:39)
 at com.ca.shift.client.RemoteInvocation.createInvocation(RemoteInvocation.java:87)

Cause

The Trellis probe on the UIM Server causes this issue and is seen anytime the Robot is restarted, including server restarts.

The Trellis probe starts up successfully but captures the following error on startup.
Note: nq_services package is used by the nfa_inventory probe.

[main, trellis] Failed to start application nq_services: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'defaultNfaOriginService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: com.ca.trellis.persist.QueryExecutor com.ca.nfa.origin.service.impl.DefaultNfaOriginService.executor; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'datomic-query-executor' defined in com.ca.trellis.datomic.QueryExecutorConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate 

Trellis probe by default is set to startup after the udm_manager but the udm_manager is not completely up and trellis reports the nq_services error.

When the Trellis probe is in this condition, the nfa_inventory log captures the error below every 15 minutes.

[interfaceToOriginMapping, nfa_inventory] Failed to execute origin mapping service, returning empty results.

Environment

Release : 9.2.0

Component : UIM - NFA_INVENTORY

Resolution

To resolve this issue, we need to give the udm_manager more startup time before the Trellis probe starts using the steps below:

1. RDP to the UIM Server (Primary hub)
2. Edit the ..\Nimsoft\robot\controller.cfg and go to the  <trellis> section
3. Change start_after = udm_manager to start_after = wasp
4. Restart the UIM Server robot from IM or Windows Services
5. Check the Trellis probe after it starts and make sure the above trellis startup nq_services error is not seen.
6. Check the nfa_inventory log  after 30+ minutes and make sure that "Failed to execute origin mapping service" is not seen.