After Operating System patching AutoSys Agent, Web Server (AEWS), SOAP Web Server, none of these start anymore

Products

• Autosys Workload Automation

Issue/Introduction

When OS patching is done, it is possible that the Java / JRE / JDK as part of the Operating System gets updated too.

When AutoSys Agent/Web Server are configured to use operating system Java, and if the Java Paths referred to are no longer valid now, neither Agent/Web Server can start.

./cybAgent -a
Agent service is starting...
The service did not respond to the start request in a timely fashion

Agent's cybagent_service.log contains an error like this

08/09/2024 10:17:21.904-0400: Database check succeeded
08/09/2024 10:17:21.904-0400: JVM doesn't exist at the specified java path /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el7_7.x86_64/jre/bin/

 

With restart option of unisrvcntr, one would see a FAILED status on the Web Server and Soap Web Server

WAAE Soap Web Server FAILED
WAAE Web Server (ACE) FAILED

 

Cause

This is because the Agent / Web Server / Soap Web Server were configured with a hard coded JVM path that does not exist on the operating system anymore.

/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el8_7.x86_64/jre/bin/

When OS patching is done, if Java/JRE packages were added as part of the OS, then those Java/JRE packages get upgraded.  Older Java/JRE builds are removed too. Which means the build 1.8.0.352 in the above case, is no longer present on the OS. 

This causes Agent, SOAP Web Server and Web Server not to start anymore.

Resolution

1. Identify the higher level Java/JRE folder instead of going for the individual build number based folder names, that contain the real JRE that the OS has.
   
   example: 
   ls -al /etc/alternatives/jre_openjdk
lrwxrwxrwx. 1 root root 75 Jul 25 11:20 /etc/alternatives/jre_openjdk-> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-2.0.1.el7.x86_64/jre

Note: the build number is 412 above, as opposed to 352 that the agent was looking for earlier
   
   
2. Use  /etc/alternatives/java  for Agent/Web Server/SOAP Web Server configurations, instead of using the real path /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-2.0.1.el7.x86_64/jre
3. Fix references to the correct Java across the product components:
   
   
   1. In Agent's agentparm.txt, look for   oscomponent.javapath=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-2.0.1.el7.x86_64/jre

      
      1. Fix the above parameter value to the correct higher level path instead
      2.  oscomponent.javapath=/etc/alternatives/jre_openjdk
      3. Save the file and restart Agent
         
         
   2. Verify if Web Server is using a hard coded Java path or if it is using Autosys's JRE folder
      1. In the file  $AUTOUSER/webserver/conf/wrapper.conf check for  set.JAVA_HOME = ../../JRE_WA

      2. As it is referring to a relative path, check if the JRE_WA  folder is a soft link or a real folder
         

         ls -al /opt/CA/WorkloadAutomationAE
lrwxrwxrwx.  1 root    root       29 Jul 25 11:22 JRE64_WA -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el7.x86_64/jre
lrwxrwxrwx.  1 root    root       29 Jul 25 11:21 JRE_WA -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el7.x86_64/jre

         Note: the above real folder doesn't exist anymore because of the OS patching.  
         
         Change the softlinks to point to correct folder that exists as documented here 
         
         rm -rf /opt/CA/WorkloadAutomationAE/JRE_WA
rm -rf /opt/CA/WorkloadAutomationAE/JRE64_WA
ln -s /etc/alternatives/jre_openjdk JRE_WA
ln -s /etc/alternatives/jre_openjdk JRE64_WA
         
         Check the folder structure again now:
         ls -al /opt/CA/WorkloadAutomationAE
lrwxrwxrwx.  1 root    root       29 Jul 25 11:22 JRE64_WA -> /etc/alternatives/jre_openjdk
lrwxrwxrwx.  1 root    root       29 Jul 25 11:21 JRE_WA -> /etc/alternatives/jre_openjdk
         
         
         
      3. Restart the Web Server / Soap Web Server