DX OI 21.3.1 doireadserver pod in crashloopback.

Products

• DX Operational Intelligence

Issue/Introduction

Symptoms:

- doireadserver pod in crashloopback.  

- Error in doireadserver pod log: 

13:22:41.636 [main] ERROR com.ca.emm.corejsvr.crypto.BouncyCastleInitializer - TID[0] class  
"org.bouncycastle.crypto.CryptoServicesRegistrar"'s signer information does not match signer information of other classes in the same  package; exception stack trace:  
java.lang.SecurityException: class "org.bouncycastle.crypto.CryptoServicesRegistrar"'s signer information does not match signer  information of other classes in the same package  
 at java.lang.ClassLoader.checkCerts(ClassLoader.java:891) ~[?:1.8.0_302]  
 at java.lang.ClassLoader.preDefineClass(ClassLoader.java:661) ~[?:1.8.0_302]  
 at java.lang.ClassLoader.defineClass(ClassLoader.java:754) ~[?:1.8.0_302]  
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[?:1.8.0_302]  
 at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[?:1.8.0_302]  
 at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[?:1.8.0_302]  
 at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[?:1.8.0_302]  

Cause

The problem is related to defect DE529741

 

 

Environment

DX Operational Intelligence 21.3.1 

Resolution

To be fixed in a next on premise release

 

Workaround: apply available hotfixes:

- If you are planning to install 21.3.1 GA build only download attached doi-readserver-21-3-1-ga-hf2.tar.gz

- If you are planning to install 21.3.1 hf1 and/or hf 2,download attached  doi-readserver-21-3-1-hf2-hf1.tar.gz

 

Extract the content using : gzip -d <doi-readserver-21-3-1-xxx.gz>

Steps to apply hotfix:  

 

Step #1: load image referencing the tar file  

For doi-readserver-21-3-1-ga-hf2.tar

docker load --input doi-readserver-21-3-1-ga-hf2.tar

 

For doi-readserver-21-3-1-hf2-hf1.tar

docker load --input doi-readserver-21-3-1-hf2-hf1.tar

 

Output:  

5e03d8cae877: Loading layer [==================================================>] 5.855MB/5.855MB  
689a1783490f: Loading layer [==================================================>] 6.668MB/6.668MB  
40f205c89aff: Loading layer [==================================================>] 10.29MB/10.29MB  
cb6796a0c6ab: Loading layer [==================================================>] 73.9MB/73.9MB  
a1d328b8bc30: Loading layer [==================================================>] 14.08MB/14.08MB  
381082c715d6: Loading layer [==================================================>] 150.6MB/150.6MB  
c7f61aee32b2: Loading layer [==================================================>] 1.731MB/1.731MB  
cb316e380bb7: Loading layer [==================================================>] 37.89kB/37.89kB  
2c8a30fd0757: Loading layer [==================================================>] 4.656MB/4.656MB  
4c2053f262a9: Loading layer [==================================================>] 52.74kB/52.74kB  
914292d329ff: Loading layer [==================================================>] 544.3kB/544.3kB  
b754f87ceb14: Loading layer [==================================================>] 348MB/348MB  

Loaded image: esd-oi-docker-release-local.artifactory-lvn.broadcom.net/doi/21.3.1.ga/readserver-doi:21.3.1.ga.5309  

Step #2: tag existing doireadserver image as an old release in the docker registry  

docker tag <private-registry-hostname>:5000/dxi/doi-21.3.1.ga-readserver-doi:21.3.1.ga.4213 <private-registry-hostname>:5000/dxi/doi-21.3.1.ga-readserver doi:old_release  

Step #3: tag new image to appropriate name  

docker tag esd-oi-docker-release-local.artifactory-lvn.broadcom.net/doi/21.3.1.hf2/readserver-doi:21.3.1.hf2.5432 <private-registry-hostname>:5000/dxi/doi 21.3.1.hf1-readserver-doi:21.3.1.hf2.5432 

Step #4: Push docker image to central repository  

docker push <private-registry-hostname>:5000/dxi/doi-21.3.1.hf1-readserver-doi:21.3.1.hf2.5432 

Output:  

The push refers to repository [<private-registry-hostname>:5000/dxi/doi-21.3.1.ga-readserver-doi] b754f87ceb14: Pushed  

914292d329ff: Pushed  
4c2053f262a9: Pushed  
2c8a30fd0757: Pushed  
cb316e380bb7: Pushed  
c7f61aee32b2: Pushed  
381082c715d6: Pushed  
a1d328b8bc30: Pushed  
cb6796a0c6ab: Pushed  
40f205c89aff: Pushed  
689a1783490f: Pushed  
5e03d8cae877: Pushed  

21.3.1.ga.5309: digest: sha256:2d0e17448f2676499bdd11a239a2a98305bba44d3d2e46004c2daf66c4b4ce0d size: 2847  

 

Step# 5: Scale down doireadserver pod  

kubectl scale --replicas=0 deployment doireadserver -n<namespace>

 

Step #6: Edit image deployment to reference new image name  

kubectl edit deployment doireadserver -n dxi 

search for "image:" and replace the tag with the latest one  

<private-registry-hostname>:5000/dxi/doi-21.3.1.ga-readserver-doi:21.3.1.ga.5432

save the deployment  

 

Step #7: Scale up the doireadserver pod  

kubectl scale --replicas=1 deployment doireadserver -n<namespace>

 

Make sure the pod comes to the running state, once it is please check for the pod logs.

 

Additional Information

https://knowledge.broadcom.com/external/article/190815/aiops-troubleshooting-common-issues-and.html#mcetoc_1fue6hbup1