uimapi and webservices_rest stop working after upgrading to 20.3.3 or 20.4: Pool not open

book

Article ID: 212941

calendar_today

Updated On:

Products

DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

After upgrading to UIM 20.3.3 REST API calls  are failing with HTTP 500 error. This is the exception as reported in the browser when attempting an API call:

java.lang.IllegalStateException: Pool not open	
org.apache.commons.pool2.impl.BaseGenericObjectPool.assertOpen(BaseGenericObjectPool.java:736)

The failure happens with both uimapi and webservices_rest deployed to the OC robot, and the uimapi log has the exact same info the browser shows. 

2021-04-13 13:07:46,922 ERROR com.ca.uimapi.services.controller.AccountResource:getAccountsV0:123 [http-nio-80-exec-20]   - getAccounts failed: 
java.lang.IllegalStateException: Pool not open
 at org.apache.commons.pool2.impl.BaseGenericObjectPool.assertOpen(BaseGenericObjectPool.java:736)
 at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:414)
 at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:365)
 at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
 at com.nimsoft.db.PooledDataSourceWrapper.getConnection(PooledDataSourceWrapper.java:33)

If using webservices_rest with a GET call direct in a URL without a REST client it might just return a http 404 error. 

When using a REST client it will show up in the return data. 


In some Scenarios, only specific QOS metrics calls, or account calls are failing when using webservices_rest while uimapi keeps working fine:

Example Error: 

ERROR [http-nio-80-exec-9, org.apache.catalina.core.ContainerBase.[wasp-engine].[localhost].[/rest].[webservices_rest]] invoke() Servlet.service() for servlet [webservices_rest] in context with path [/rest] threw exception
toukokuuta 19 12:40:00:451 ERROR [http-nio-80-exec-9, org.apache.catalina.core.ContainerBase.[wasp-engine].[localhost].[/rest].[webservices_rest]] java.lang.IllegalStateException: Pool not open
 at org.apache.commons.pool2.impl.BaseGenericObjectPool.assertOpen(BaseGenericObjectPool.java:736)
 at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:414)
 at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:365)
 at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
 at com.nimsoft.db.PooledDataSourceWrapper.getConnection(PooledDataSourceWrapper.java:33)
 at com.nimsoft.nms.nimwscommon.dao.Util.getConnection(Util.java:45)
 at com.nimsoft.nms.nimwscommon.controller.QoSController.getConnection(QoSController.java:47)
 at com.nimsoft.nms.restfulws.services.QosResource.getDefinition(QosResource.java:62)
 at sun.reflect.GeneratedMethodAccessor1137.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
 at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
 at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)

Cause

The failure is caused by old packages at the OC robot. 

Compare the installed packages to this list:
uim_reportscheduler 20.33hf2
ump 20.33
ump_accountadmin 20.33hf2
ump_cabi 4.24hf2
ump_dashboard 20.33hf1
ump_operatorconsole 2.10hf3
ump_slm 20.33hf2 
wasp 20.33

Any that are version 9.x, or 8.x are to be suspected as deprecated and should be removed. 

webapps known to cause the problem: dapi, jasperserver_pro, umpjslib, webalarms, HtmlAlarmConsole, umpmeadia, maintenancemode (OC/ump webapp)

Note: jasperserver_pro will only show up in wasp raw configure or cfg under the webapps section. 

Environment

Release : UIM 20.3.x, UIM 20.4

Component : UIM - WEBSERVICES_REST

Resolution

Remove the old packages from the OC robot. 

UIM How to remove deprecated webapps from wasp
Article Id: 212899
https://knowledge.broadcom.com/external/article?articleId=212899

Additional Information

uimapi log location: OC robot > $)\Nimsoft\probes\service\wasp\uimapi.log 

UIM / DX IM - How to see all the packages deployed to a robot
Article Id: 141322
https://ca-broadcom.wolkenservicedesk.com/external/article?articleId=141322

The obsolete webapps are documented here:
20.3 Operator Console Installer - Webapps Considerations

Attachments