Hi Team,
I am trying to upgrade one of the registry server to servicepack 3(SP3), On trying to logging on the portal i see below error message getting displayed:
An error occurred while retrieving the list of VSE servers.
Error 503 - <html><head><meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/><title>Error 503 Service Unavailable</title></head><body><h2>HTTP ERROR 503 Service Unavailable</h2><table><tr><th>URI:</th><td>/lisa-test-invoke/api/v1/serverstatuses</td></tr><tr><th>STATUS:</th><td>503</td></tr><tr><th>MESSAGE:</th><td>Service Unavailable</td></tr><tr><th>SERVLET:</th><td>-</td></tr></table></body></html>
In portal logs below error is displayed:
2023-09-28T14:24:17,066Z (10:24) [qtp773896351-16] ERROR org.grails.web.errors.GrailsExceptionResolver - RetrofitError occurred when processing request: [GET] /devtest/api/sv/vseServers/name,%20performanceEnabled,serviceFQDN/all
503 Service Unavailable. Stacktrace follows:
retrofit.RetrofitError: 503 Service Unavailable
at retrofit.RetrofitError.httpError(RetrofitError.java:39) ~[retrofit-1.6.1.jar:?]
Registry log error:
2023-09-28T14:20:25,142Z (10:20) [Thread-13] ERROR org.springframework.boot.SpringApplication - Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'mongoMetricsConnectionPoolListener' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/mongo/MongoMetricsAutoConfiguration$MongoConnectionPoolMetricsConfiguration.class]: Unsatisfied dependency expressed through method 'mongoMetricsConnectionPoolListener' parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mongoConnectionPoolTagsProvider' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/mongo/MongoMetricsAutoConfiguration$MongoConnectionPoolMetricsConfiguration.class]: Post-processing of merged bean definition failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.micrometer.core.instrument.binder.mongodb.DefaultMongoConnectionPoolTagsProvider] from ClassLoader [WebAppClassLoader{2024959361}@78b26d81]
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) ~[spring-beans-5.3.27.jar:5.3.27]
VSE log error:
2023-09-28T14:26:07,404Z (10:26) [DownloadEDLRAckNack [VS_DownloadEDLRAckNack_Run]/1] ERROR com.itko.lisa.test.TestNode - TestNode caught an exception: java.util.ConcurrentModificationException
com.itko.lisa.test.TestRunException: java.util.ConcurrentModificationException
at com.itko.lisa.vse.stateful.protocol.http.HttpListenStep.execute(HttpListenStep.java:623) ~[lisa-virtualize-10.7.2.jar:?]
2023-09-28T14:26:07,404Z (10:26) [DownloadEDLRAckNack [VS_DownloadEDLRAckNack_Run]/1] ERROR com.itko.lisa.test.TestCase - Caught a throwable; raising EVENT_TESTRUNERROR
com.itko.lisa.test.TestRunException: java.util.ConcurrentModificationException
Release : 10.7.2
Mongo DB driver jar was in the hotDeploy folder , this was interfering with the lisa jars.
In the Registry logs, we saw errors related to MongoDB.
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'mongoMetricsConnectionPoolListener' defined in class path resource [org/springframework/boot/actuate/autoconfigure/metrics/mongo/MongoMetricsAutoConfiguration$MongoConnectionPoolMetricsConfiguration.class]: Unsatisfied dependency expressed through method 'mongoMetricsConnectionPoolListener' parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mongoConnectionPoolTagsProvider' defined in class path resource
We noticed that there was a MongoDB driver jar in the hotDeploy folder. WE restarted Registry after removing that jar file. and Now Registry come up without any errors. We then started Portal and VSE.
The driver jars files should only be in /lib/shared folder... Not in hotDeploy.
we stopped all the components , cleaned the logs folder. Then Started Registry.