DE512163 - NetOps is extreamly slow, dashboards don't populate the data, lots of timeouts in the web interface

book

Article ID: 222560

calendar_today

Updated On:

Products

CA Performance Management - Usage and Administration

Issue/Introduction

NetOps is EXTREAMLY slow, dashboards dont polulate data, lots of timeouts in the web aftert upgrade to 21.2 
Dashboards timeout after upgrade to 21.2
No data in dashboards after upgrade to 21.2 

Cause

The cause of the issue is bean.xml is getting loaded before cfx.xml. To determine if you are encountering this issue check the load order and see if the bean.xml is getting loaded before cxf.xml in the PCService.log. You should restart caperfcenter_console to make sure the startup entries are in the current PCService.log. Example below: 

INFO  | WrapperSimpleAppMain     | 2021-08-23 15:05:19,324 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from ServletContext resource [/META-INF/spring/bean.xml].      <-- BAD
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:05:19,414 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from ServletContext resource [/META-INF/spring/webservices.xml]
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:05:19,509 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from class path resource [META-INF/cxf/cxf.xml]      <-- BAD

Environment

Release : 21.2

Component : CA Performance Center

Resolution

The solution is to rename /opt/CA/PerformanceCenter/PC/webapps/pc/META-INF/spring/bean.xml to application.xml (in the same location) and then restart the PC Services.  As long as the CXF is loaded before our application config data (application.xml, the increased timeout setting should be respected.

DETAILS/Verification
The following is output in PCService.log before renaming bean.xml to application.xml. You can see that bean.xml gets loaded before cxf.xml, which means that CXF missed the updated timeout because the bean.xml has already been processed before cxf could internalize it.

INFO  | WrapperSimpleAppMain     | 2021-08-23 15:05:19,324 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from ServletContext resource [/META-INF/spring/bean.xml].   <-- BAD
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:05:19,414 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from ServletContext resource [/META-INF/spring/webservices.xml]
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:05:19,509 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from class path resource [META-INF/cxf/cxf.xml] <-- BAD

The following is an output after renaming bean.xml to application.xml. You can see that application.xml gets loaded after cxf.xml, which is needed for the updated timeout to take effect.  CXF is loaded into memory and can internalize the config found in the application.xml.

INFO  | WrapperSimpleAppMain     | 2021-08-23 15:26:47,548 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from class path resource [META-INF/cxf/cxf.xml]. <-- GOOD
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:26:47,575 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from class path resource [META-INF/cxf/cxf-servlet.xml]
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:26:48,054 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from ServletContext resource [/META-INF/spring/rib-support.xml]
INFO  | WrapperSimpleAppMain     | 2021-08-23 15:26:48,068 | org.springframework.beans.factory.xml.XmlBeanDefinitionReader
      | Loading XML bean definitions from ServletContext resource [/META-INF/spring/application.xml] <-- GOOD

Additional Information

Fix will be in release 21.2.3