Restarting the SSH Daemon (SSHD) in the MOI DSP Docker Container for Mbuild to run as scheduled
search cancel

Restarting the SSH Daemon (SSHD) in the MOI DSP Docker Container for Mbuild to run as scheduled

book

Article ID: 230831

calendar_today

Updated On:

Products

Mainframe Operational Intelligence

Issue/Introduction

The MOI DSP docker container uses an SSH Daemon to talk to the world outside the DSP docker container and run utility processes that are needed for machine learning.   

If the SSHD fails for some reason, these utility processes will not run. 

As an example of the impact of the SSHD The Mbuild process that builds a new machine learning model is scheduled to run every day.     If the SSHD dies for some unknown reason, Mbuild will not run.   If Mbuild does not run for a lengthy period of time, the Green Highways in MOI will revert back to the EWMA learning mode and no alerting will occur in MOI.   The SSHD has to be restarted and the Mbuild will have to run for a period of a week or more to switch back to KDE mode and resume alerting.   

The steps to restart the SSHD are contained in the Resolution of this Knowledge Doc. 

 

Environment

Mainframe Operational Intelligence 2.0

Release 2.0.06

Resolution

How to restart the SSH Daemon to ensure that the dsp machine learning microservice utility processes run as scheduled: 

 

Check the status of the dsp docker container: 

docker ps -a|grep dsp


c64b93c519a1        mtcoi.packages.ca.com/x86_64/dsp:02.0.06-202111161001                            "/bin/sh -c '[[ -f..."   3 days ago          Up 3 days (healthy)  

 Stop the dsp docker container: 

cd /opt/moi

bin/plex down -- dsp

Start the dsp docker container: 

bin/plex up -d -- dsp

Ensure that the dsp microservice becomes healthy after 5 to 10 mins: 

docker ps -a | grep  dsp

If the Mbuild process is the one that didn't run due to the SSHD dying, the next day or day and a half after stopping and starting the dsp docker container, run these commands that verify that the machine learning model build (MBuild) is running successfully on a scheduled basis which indicates that the SSH Daemon is alive:  

To run a quick check on the status of the MBuild process in MOI, run the following two commands:

ls -la /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log*

zgrep "Successfully" /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log*

MOI Support may request that the output of these two commands be uploaded to an open MOI Support Case. 

Example command output from your MOI appliance that your commands should produce: 

[root@oi-n-plex-cho03 dsp_logs]# ls -la /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log*
-rw-r--r--. 1 ca_itoa ca_analytics 1332 Dec  9 10:19 /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211209-091553.gz
-rw-r--r--. 1 ca_itoa ca_analytics 1336 Dec  9 23:08 /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211209-215029.gz
-rw-r--r--. 1 ca_itoa ca_analytics 1335 Dec 10 23:00 /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211210-215031.gz
-rw-r--r--. 1 ca_itoa ca_analytics 1334 Dec 11 22:46 /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211211-215029.gz
-rw-r--r--. 1 ca_itoa ca_analytics 1333 Dec 12 22:55 /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211212-215035.gz

[root@oi-n-plex-cho03 dsp_logs]# zgrep "Successfully finished NightlyMBuild" /var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log*
/var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211209-091553.gz:ADMIN: 2021-12-09-10:19:56.829 P01866/Thr00 Successfully finished NightlyMBuild
/var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211209-215029.gz:ADMIN: 2021-12-09-23:08:20.854 P09364/Thr00 Successfully finished NightlyMBuild
/var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211210-215031.gz:ADMIN: 2021-12-10-23:00:07.501 P05615/Thr00 Successfully finished NightlyMBuild
/var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211211-215029.gz:ADMIN: 2021-12-11-22:46:34.009 P01449/Thr00 Successfully finished NightlyMBuild
/var/opt/moi/data2/profiler/var/opt/CA/itoa/dsp/dsp_logs/mbuild.log.20211212-215035.gz:ADMIN: 2021-12-12-22:55:46.243 P25315/Thr00 Successfully finished NightlyMBuild

 

Powered by Wolken Software