search cancel

Upgrade UIM to 20.3 on Linux fails on Configuring Data Engine

book

Article ID: 203110

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The upgrade from UIM 9.x/20.1 to 20.3 on Linux is stuck at deploying data_engine:

 

[1/53] Configuring core server components ... - OK
- OK
[3/53] Configuring primary robot - 1 ... - OK
[4/53] Configuring primary robot - 2 ... - OK
[5/53] Configuring primary robot - 3 ... - OK
[6/53] Configuring primary primary hub probe ... - OK
[7/53] Configuring distsrv probe ... - OK
[8/53] Starting server ... - OK
[9/53] Configuring security - 1 ... - OK
[10/53] Configuring security - 2 ... - OK
[11/53] Configuring probe licenses ... - OK
[12/53] Configuring server for package deployment ... - OK
[13/53] Configuring Java Runtime Environment ... - OK
[14/53] Configuring Automated Deployment Engine probe ... - OK
[15/53] Configuring Data Engine probe ...

 

2020-11-02 16:02:13,949 DEBUG probe.AdeController:distributePackageSynchronous:119 [Thread-57]  - deploy_probe request sent for pkg: data_engine, version: null
2020-11-02 16:02:18,950 DEBUG probe.AdeController:distributePackageSynchronous:138 [Thread-57]  - Sending getPackageDistributionStatus.
2020-11-02 16:02:18,951 DEBUG pds.PDSController:sendPDSWithAddr:133 [Thread-57]  - Sending 'get_status' with sid: FL..., timeout: 120000
2020-11-02 16:02:18,965 DEBUG pds.PDSController:sendPDSWithAddr:138 [Thread-57]  - Command 'get_status' response retrieved.
 JobStatus=Running
 JobDescription=Automated deployment job Created at 1604332933929
 EndTime=0.0
 JobName=automated_deployment-1604332933929
 StartTime=373.2310132524

 StatusTable (PDS)

  /MDS/mdsmonl1/mdsmonl1/data_engine (PDS)
    Status=Running
    Type=Probe
    TaskId=2
    Description=
    Address=
    Version=20.30
    Error=
    Host=/MDS/mdsmonl1/mdsmonl1
    Package=data_engine
    JobID=3e96894e3b6c4df1a183c437906c6faa
 JobID=3e96894e3b6c4df1a183c437906c6faa
2020-11-02 16:02:18,966 DEBUG probe.AdePackageDistributionStatus:parsePDS:48 [Thread-57]  - &&&&Task PDS:             /MDS/mdsmonl1/mdsmonl1/data_engine: Status=Running, Type=Probe, TaskId=2, Description=, Address=, Version=20.30, Error=, Host=/MDS/mdsmonl1/mdsmonl1, Package=data_engine, JobID=3e96894e3b6c4df1a183c437906c6faa
2020-11-02 16:02:18,966 DEBUG probe.AdePackageDistributionStatus:parsePDS:76 [Thread-57]  - $$$$$$$$$Inside parsePDS: Running -- running
2020-11-02 16:02:18,967 DEBUG pds.PDSController:sendPDSWithAddr:133 [Thread-57]  - Sending 'archive_list' with sid: FL..., timeout: 120000
2020-11-02 16:02:18,972 DEBUG pds.PDSController:sendPDSWithAddr:138 [Thread-57]  - Command 'archive_list' response retrieved.
2020-11-02 16:02:18,973 DEBUG probe.AdeController:getPackageDistributionStatus:314 [Thread-57]  - ADE distribution_status - is_successful: false, AdePackageDistributionStatus{strJobId=3e96894e3b6c4df1a183c437906c6faa, strJobName=automated_deployment-1604332933929, strJobDescription=, strPackageName=data_engine, strPackageVersion=20.30, strRobotAddress=/MDS/mdsmonl1/mdsmonl1, strStatus=Running, startTime=1604332933932, endTime=0, strTaskId=2, strTaskStatus=running, strResultCode=-1, strResultString=Failed, strInstState=Not Deployed, strAttempts=1, strType=Probe, strDescription=Data Engine, strCopyRight=Copyright (c) 2020, CA. All rights reserved., strGroup=SLM, strAuthor=CA, strDate=12.08.2020, strBuild=71, strFileName=/opt/nimsoft/archive/data_engine.zip, strLicenseRqd=no}
2020-11-02 16:02:18,974 INFO  probe.AdeController:distributePackageSynchronous:142 [Thread-57]  - distStatus.isFinished: false
2020-11-02 16:02:21,474 DEBUG probe.AdeController:distributePackageSynchronous:138 [Thread-57]  - Sending getPackageDistributionStatus.
2020-11-02 16:02:21,474 DEBUG pds.PDSController:sendPDSWithAddr:133 [Thread-57]  - Sending 'get_status' with sid: FL..., timeout: 120000
2020-11-02 16:02:21,478 ERROR impl.UIMServerConfigureController:run:477 [Thread-57]  - NimException caught
(80) Session error, Unable to open a client session for :48000: Connection refused (Connection refused)
 at com.nimsoft.nimbus.NimClientSession.<init>(NimClientSession.java:66)
 at com.nimsoft.nimbus.NimNamedClientSession.<init>(NimNamedClientSession.java:27)
 at com.nimsoft.nimbus.NimObjectSender.connectNamed(NimObjectSender.java:78)
 at com.nimsoft.nimbus.NimRequest.sendImpersonate(NimRequest.java:252)
 at com.nimsoft.install.nimcommon.pds.PDSController.sendRequest(PDSController.java:213)
 at com.nimsoft.install.nimcommon.pds.PDSController.sendPDSWithAddr(PDSController.java:136)
 at com.nimsoft.install.nimcommon.pds.PDSController.sendPDSWithAddr(PDSController.java:113)
 at com.nimsoft.install.nimcommon.pds.PDSController.sendWithAddr(PDSController.java:108)
 at com.nimsoft.install.nimcommon.probe.AdeController.getPackageDistributionStatus(AdeController.java:305)
 at com.nimsoft.install.nimcommon.probe.AdeController.distributePackageSynchronous(AdeController.java:139)
 at com.nimsoft.install.nimcommon.probe.AdeController.distributePackageSynchronous(AdeController.java:99)
 at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.distributePackageAde(UIMServerConfigureController.java:3080)
 at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.distributePackageCommon(UIMServerConfigureController.java:3027)
 at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.configureDataEngine(UIMServerConfigureController.java:1004)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.nimsoft.install.nimcommon.method.NimMethodCallback.invokeMethod(NimMethodCallback.java:532)
 at com.nimsoft.install.nimcommon.method.NimMethodCallback.invokeMethod(NimMethodCallback.java:516)
 at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.doConfigure_postNMSStartupPackages(UIMServerConfigureController.java:530)
 at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.doConfigure(UIMServerConfigureController.java:491)
 at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.run(UIMServerConfigureController.java:475)
 at java.lang.Thread.run(Thread.java:748)

 

 

- Deleting ADE folder totally and retry upgrade still fails

- Deploying data_engine all alone same version also fails

- Downgrading ADE and deploying data_engine current version also fails with the same error. 

 

11032020 15:55:13,564 [pool-6-thread-1] DEBUG RobotFileInstallerStream - Starting installation of file data_engine for package data_engine with PDS     mode=0755, package=data_engine, file=data_engine, type 9 not found, type=binary 
11032020 15:55:13,571 [pool-6-thread-1] TRACE RobotFileInstallerStream - Return value for install of file data_engine for cmd inst_file_start     id=19, written=1048576 
11032020 15:55:13,571 [pool-6-thread-1] DEBUG RobotFileInstallerStream - id for file transfer of data_engine for package data_engine is 19 
11032020 15:55:13,615 [pool-6-thread-1] TRACE RobotFileInstallerStream - Return value for install of file data_engine for cmd inst_file_next     id=19, written=2097152 
11032020 15:55:13,622 [pool-6-thread-1] TRACE RobotFileInstallerStream - Return value for install of file data_engine for cmd inst_file_next     id=19, written=3145728 
11032020 15:55:13,629 [pool-6-thread-1] TRACE RobotFileInstallerStream - Return value for install of file data_engine for cmd inst_file_next     id=19, written=4194304 
11032020 15:55:13,637 [pool-6-thread-1] TRACE RobotFileInstallerStream - Return value for install of file data_engine for cmd inst_file_next     id=19, written=5242880 
11032020 15:55:13,640 [pool-6-thread-1] TRACE RobotFileInstallerStream - Return value for install of file data_engine for cmd inst_file_next     id=19, written=5538184 
11032020 15:55:13,640 [pool-6-thread-1] DEBUG NimbusTransport - [Addr: /MDS/mdsmonl1/mdsmonl1] File transfer for data_engine for package data_engine complete. 
11032020 15:55:13,670 [pool-6-thread-1] DEBUG RobotFileInstallerStream - Closed the file transfer for file data_engine for package data_engine. Got result PDS from 'inst_file_end'     id=19, written=5538184 
11032020 15:55:13,670 [pool-6-thread-1] INFO  ProbeStrategy$TransferSections - [instFile] File transfer finished -> File { data_engine binary 0755} Target /MDS/mdsmonl1/mdsmonl1. Attempts: 1 
11032020 15:55:13,670 [pool-6-thread-1] INFO  ProbeStrategy$TransferSections - Waiting for controller to become ready for section LINUX_23_64 of package data_engine 
11032020 15:55:13,670 [pool-6-thread-1] DEBUG ProbeStrategy$TransferSections - Request timeout set to : 0 
11032020 15:55:13,671 [pool-6-thread-1] DEBUG NimbusTransport - NimbusTransport() Nimrequest timeout : 0 
11032020 15:55:13,871 [pool-6-thread-1] INFO  ProbeStrategy$TransferSections - Calling 'inst_execute' on section 'LINUX_23_64' for package 'data_engine' 
11032020 15:55:13,872 [pool-6-thread-1] DEBUG NimbusTransport - NimbusTransport() Nimrequest timeout : 180000 
11032020 15:55:14,606 [attach_socket] TRACE AutomatedDeploymentEngineProbe - Retrieving status message(s) for 3352d369e61442d8b3dafd82db13cd79 using Locale(en_US) 
11032020 15:55:14,607 [attach_socket] TRACE ADEH2Dal - Retrieved DBJob from database -> DBJob{jobId='3352d369e61442d8b3dafd82db13cd79', jobName='automated_deployment-1604418910280', jobDescription='Automated deployment job Created at 1604418910281', startTime=1604418910285, stopTime=0, jobStatus='Running'} 
11032020 15:55:14,608 [attach_socket] TRACE TrackerUtil - -- createPSDForDBTask -- retrieving status message using Locale(en_US) 
11032020 15:55:14,608 [attach_socket] TRACE TrackerUtil - -- createPDSForDBTask -- retrieving status message for using Locale(en_US) failed. Using default Locale(en_US) 
11032020 15:55:14,608 [attach_socket] TRACE AutomatedDeploymentEngineProbe - ADE - get_status callback PDS dump:  

 

 

iaoutput:

2021-02-04 15:43:17,954 ERROR console.NimCustomCodeConsoleBase:executeConsoleAction:134 [main]  - Exception caught during processing. Aborting installation.
(80) Session error, Unable to open a client session for :48000: Connection refused (Connection refused)
    at com.nimsoft.nimbus.NimClientSession.<init>(NimClientSession.java:66)
    at com.nimsoft.nimbus.NimNamedClientSession.<init>(NimNamedClientSession.java:27)
    at com.nimsoft.nimbus.NimObjectSender.connectNamed(NimObjectSender.java:78)
    at com.nimsoft.nimbus.NimRequest.sendImpersonate(NimRequest.java:252)
    at com.nimsoft.install.nimcommon.pds.PDSController.sendRequest(PDSController.java:213)
    at com.nimsoft.install.nimcommon.pds.PDSController.sendPDSWithAddr(PDSController.java:136)
    at com.nimsoft.install.nimcommon.pds.PDSController.sendPDSWithAddr(PDSController.java:113)
    at com.nimsoft.install.nimcommon.pds.PDSController.sendWithAddr(PDSController.java:108)
    at com.nimsoft.install.nimcommon.probe.AdeController.getPackageDistributionStatus(AdeController.java:305)
    at com.nimsoft.install.nimcommon.probe.AdeController.distributePackageSynchronous(AdeController.java:139)
    at com.nimsoft.install.nimcommon.probe.AdeController.distributePackageSynchronous(AdeController.java:99)
    at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.distributePackageAde(UIMServerConfigureController.java:3080)
    at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.distributePackageCommon(UIMServerConfigureController.java:3027)
    at com.nimsoft.install.uimserver.action.impl.UIMServerConfigureController.configureDataEngine(UIMServerConfigureController.java:1004)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)

 

Environment

Release : 20.3

Component : UIM - INSTALL

UIM 20.1 > Upgrade to 20.3
ON: Red Hat Enterprise Linux Server release 7.4 (Maipo) / CentOS
The DB is MySQL 5.6 on a separate machine ON: Red Hat Enterprise Linux Server release 7.4 (Maipo)

Cause

A bad formatted environment variable was preventing the data_engine from being deployed. due to this corrupt variable the probe wouldn't deploy and would stuck the upgrade.

in the UIM server variables in controller looked like:

      LD_LIBRARY_PATH = .:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::$LD_LIBRARY_PATH
      NIM_JRE_HOME = jre/jre8u232b09
      NIM_JRE_HOME_1_8 = jre/jre8u232b09

 

Resolution


Remove the corrupted entries from the LD_LIBRARY_PATH key located in the robot.cfg.

(or set the LD_LIBRARY_PATH key to a blank value)

 

Examples:

 

If you have something like:

LD_LIBRARY_PATH = ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.:/opt/oracle/instantclient_12_1:$LD_LIBRARY_PATH

 

Change it to:

LD_LIBRARY_PATH = .:/opt/oracle/instantclient_12_1:$LD_LIBRARY_PATH

 

 

Or If you have something like:

LD_LIBRARY_PATH = .:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::$LD_LIBRARY_PATH

 

Change it to:

LD_LIBRARY_PATH =

 

Restart the robot and try again the upgrade. After this the data_engine should be installed and the upgrade should complete succesfully.

 

 

Additional Information

Also encountered in a 9.02 -> 9.20 UIM upgrade installation where the LD_LIBARY_PATH was corrupted.