DX AIOps 21.3.1 - Unable to login - doireadserver pod in crashloopback

Products

• DX Operational Intelligence

Issue/Introduction

Symptoms:

- doireadserver pod is in crashloopback.  

- The following error is reported in doireadserver pod log: 

13:22:41.636 [main] ERROR com.ca.emm.corejsvr.crypto.BouncyCastleInitializer - TID[0] class  
"org.bouncycastle.crypto.CryptoServicesRegistrar"'s signer information does not match signer information of other classes in the same  package; exception stack trace:  
java.lang.SecurityException: class "org.bouncycastle.crypto.CryptoServicesRegistrar"'s signer information does not match signer  information of other classes in the same package  
 at java.lang.ClassLoader.checkCerts(ClassLoader.java:891) ~[?:1.8.0_302]  
 at java.lang.ClassLoader.preDefineClass(ClassLoader.java:661) ~[?:1.8.0_302]  
 at java.lang.ClassLoader.defineClass(ClassLoader.java:754) ~[?:1.8.0_302]  
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_302]  
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_302]  
 at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_302]  
 at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_302]  

Environment

DX Platform 21.3.1, 21.3.1HF1

Cause

The problem is related to defect DE529741

 

 

Resolution

Solution:

Fixed in 22.1 and onward releases.

 

Workaround:

1) Download one of the attached package :

- For 21.3.1 GA build : download attached doi-readserver-21-3-1-ga-hf2.tar.gz

- For 21.3.1 hf1, hf 2 : download attached  doi-readserver-21-3-1-hf2-hf1.tar.gz

2) Extract the content :

gzip -d <doi-readserver-21-3-1-xxx.gz>

 

3) load image:

- For 21.3.1 GA build :

docker load --input doi-readserver-21-3-1-ga-hf2.tar

 

- For 21.3.1 hf1, hf 2 :

docker load --input doi-readserver-21-3-1-hf2-hf1.tar

 

Output example:  

5e03d8cae877: Loading layer [==================================================>] 5.855MB/5.855MB  
689a1783490f: Loading layer [==================================================>] 6.668MB/6.668MB  
40f205c89aff: Loading layer [==================================================>] 10.29MB/10.29MB  
cb6796a0c6ab: Loading layer [==================================================>] 73.9MB/73.9MB  
a1d328b8bc30: Loading layer [==================================================>] 14.08MB/14.08MB  
381082c715d6: Loading layer [==================================================>] 150.6MB/150.6MB  
c7f61aee32b2: Loading layer [==================================================>] 1.731MB/1.731MB  
cb316e380bb7: Loading layer [==================================================>] 37.89kB/37.89kB  
2c8a30fd0757: Loading layer [==================================================>] 4.656MB/4.656MB  
4c2053f262a9: Loading layer [==================================================>] 52.74kB/52.74kB  
914292d329ff: Loading layer [==================================================>] 544.3kB/544.3kB  
b754f87ceb14: Loading layer [==================================================>] 348MB/348MB  

Loaded image: esd-oi-docker-release-local.artifactory-lvn.broadcom.net/doi/21.3.1.ga/readserver-doi:21.3.1.ga.5309  

 

NOTE: In the below commands replace:

<namespace>                       = with your namespace or project name
<private-registry-hostname> = with your docker registry hostname

 

4) tag image

docker tag esd-oi-docker-release-local.artifactory-lvn.broadcom.net/doi/21.3.1.hf2/readserver-doi:21.3.1.hf2.5432 <private-registry-hostname>:5000/<namespace>/doi-21.3.1.hf1-readserver-doi:21.3.1.hf2.5432 

5) Push docker image to central repository  

docker push <private-registry-hostname>:5000/<namespace>/doi-21.3.1.hf1-readserver-doi:21.3.1.hf2.5432 

 

Output example:  

The push refers to repository [<private-registry-hostname>:5000/dxi/doi-21.3.1.ga-readserver-doi] b754f87ceb14: Pushed  

914292d329ff: Pushed  
4c2053f262a9: Pushed  
2c8a30fd0757: Pushed  
cb316e380bb7: Pushed  
c7f61aee32b2: Pushed  
381082c715d6: Pushed  
a1d328b8bc30: Pushed  
cb6796a0c6ab: Pushed  
40f205c89aff: Pushed  
689a1783490f: Pushed  
5e03d8cae877: Pushed  

21.3.1.ga.5309: digest: sha256:2d0e17448f2676499bdd11a239a2a98305bba44d3d2e46004c2daf66c4b4ce0d size: 2847  

 

7) Scale down doireadserver pod  

kubectl scale --replicas=0 deployment doireadserver -n<namespace>

 

8) Edit image deployment to reference new image name  

kubectl edit deployment doireadserver -n dxi 

search for "image:" and replace the value with the new image name, for example:

  ...

 image: <private-registry-hostname>:5000/<namespace>/doi-21.3.1.ga-readserver-doi:21.3.1.ga.5432

 ...

save the deployment  

 

9) Scale up the doireadserver pod  

kubectl scale --replicas=1 deployment doireadserver -n<namespace>

 

10) Verify doireadserver pod started successfully 

Additional Information

https://knowledge.broadcom.com/external/article/190815/aiops-troubleshooting-common-issues-and.html#mcetoc_1fue6hbup1