Process Automation not Starting after Upgrade

Products

• CA Process Automation Base

Issue/Introduction

Upgraded the first node in the cluster, which the installation.log indicated that the install was a success. The node started, and logins were successful.
Upgraded the second node in the cluster, and again the installation.log indicated that the install was successful. However, the second node would not start and there were several errors in the c2o.log file indicating that there were missing jar files.

Specifically errors such as:

java.lang.ClassNotFoundException: org.bouncycastle.jce.provider.BouncyCastleProvider

 

Environment

Linux or Windows

Cause

Several, but not all, jar files were missing from the node 1 installation under /PAM/server/c2o/.c2orepository/.c2oserverresources/lib/

This folder is specifically used for mirroring the files to the cluster installations. If some of the files are missing, then the new cluster install won't ever get them and subsequently will not be able to start up. 

Resolution

To resolve the issue, a reinstall of the primary node must be performed.

Verify that the following jars exist in the /PAM/server/c2o/.c2orepository/.c2oserverresources/lib/ folder:

bcprov-jdk15on-1.52.jar
jetty-websocket-8.1.19.v20160209.jar
slf4j-api-1.7.2.jar
jackson-module-jaxb-annotations-2.8.4.jar

The versions may be different depending on the version of Process Automation, but the file names will be the same. You may see for instance jetty-websocket-8.1.10.jar instead of jetty-websocket-8.1.19.v20160209.jar - but it's the jetty-websocket that you are looking for.

If these do not exist in this mirroring folder on the primary node, a reinstall of the primary node is required.

Additional Information

From the new secondary cluster, you can also open a web browser and check the following URLs. Under normal circumstances, the jar file will be downloaded to the server, however if the files are missing you will get a 404 error - meaning, the server (node1) was reachable, but the requested resource was not found.

http://node1_server:8080/c2orepository/.c2oserverresources/lib/bcprov-jdk15on-1.52.jar 
http://node1_server:8080/c2orepository/.c2oserverresources/lib/jetty-websocket-8.1.19.v20160209.jar 
http://node1_server:8080/c2orepository/.c2oserverresources/lib/slf4j-api-1.7.2.jar 
http://node1_server:8080/c2orepository/.c2oserverresources/lib/jackson-module-jaxb-annotations-2.8.4.jar