How to enable Spectrum debug for the Performance Management integration

Products

• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

When standard logging doesn't provide enough information to determine the root cause for a problem, enabling debug may be necessary.

This will detail the steps necessary to enable, manage and disable the Spectrum OneClick Tomcat web server debug for the Performance Management integration.

Helps to determine possible causes for various issues related to the product integration

Environment

All supported Spectrum releases integrated with all supported Performance Management releases

Resolution

Enable Debug

Option A: Through the OneClick Administration web UI in a browser.

1. Launch the Spectrum OneClick tomcat web servers administration page in web browser. Default is <OC_HostName>:80.
2. Log in as the Spectrum administrative user.
3. Go to the Administration page.
4. Just below the Administration option, in the horizontal grey bar, select the Debugging option.
5. From the grey bar on the left side select the "Web Server Debug Page (Runtime)" option.
6. There are three debug options for the CA Performance Management to CA Spectrum integration listed in the Debug Controller list.
   
   1. Performance Center Integration
   2. Performance Center Integration Alarm Sync Tracking
   3. Performance Center Integration Sync
7. If the problems with the integration are related to Alarm and Event synchronization enable all three options.
8. If the problems with the integration are related to synchronization issues enable the "Performance Center Integration" & "Performance Center Integration Sync" options.
9. At the bottom of the page ensure the Current Debug Level is set to MAX.
10. Select the Apply button.

Option B: Use when debug is needed through a tomcat restart. Often useful for debugging tomcat full sync to Portal upon tomcat restart.

1. Navigate to the $SPECROOT/tomcat/webapps/spectrum/WEB-INF directory to edit the web.xml file.
2. Make a backup copy of the file. Unix command example:
   • cp web.xml web.xml.orig<date>
3. Open the web.xml for editing and find this section.
   

   <param-value>NetqosEventLog@[e-mail]@off;     NetqosAlarmSyncTrackingLog@[e-mail]acking@off;
   NetqosSyncLog@[e-mail]@off;</param-value>

   • Comparing to the administration page choices these map to:
     • PCIntegration in first line matches to UI labeled option "Performance Center Integration"
     • PCIntegrationAlarmSyncTracking in second line matches to UI labeled option "Performance Center Integration Alarm Sync Tracking
     • PCIntegrationSync in third line matches to UI labeled option "Performance Center Integration Sync"
4. Enable the options required by changing the word 'off' at the end of the line to on. Example with only PCIntegrationSync turned on.
   

   <param-value>NetqosEventLog@[e-mail]@off;     NetqosAlarmSyncTrackingLog@[e-mail]acking@off;
   NetqosSyncLog@[e-mail]@on;</param-value>

5. Save the changes to the file. When tomcat is restart the debug will be enabled and will persist through future restarts.
6. Restart tomcat to generate the necessary debug data.
7. When the data has been generated disable the debug.
   • Note web.xml enabled debug is persistent debug.
   • Failing to disable it could lead to logs filling the system disk, leading to an outage. changing any value set to on back to off.

Managing Debug

• This will generate logging in the Spectrum OneClick tomcat web servers log files. In Unix review the catalina.out file, in Windows review the stdout.log file.
• If leaving this enabled long term, monitor log file growth to ensure unexpected disk full outages aren't encountered.
• If necessary, to reset the log file and have it rotated, restart the tomcat web server service.
• Note that that step will disable the debug. Ensure it's enabled again after the web server is restarted.

Disabling Debug

Option A: Disable debug enabled through the UI.

1. Launch the Spectrum OneClick tomcat web servers administration page in web browser. Default is <OC_HostName>:80.
2. Log in as the Spectrum administrative user.
3. Go to the Administration page.
4. Just below the Administration option, in the horizontal grey bar, select the Debugging option.
5. From the grey bar on the left side select the "Web Server Debug Page (Runtime)" option.
6. At the bottom of the Debug Controller List page select the "All Off" button.

Option B: Disable debug via service restart.

• Disabling debug via Option A disables further logging but won't rotate the logs if they have become large.
• To disable the debug and rotate the logs at the same time, restart the restart the tomcat web server service.

Option c: Disable debug enabled through web.xml file edits.

• Enabling debug through web.xml file edits allows debug to persist through restarts of the tomcat service. Failure to disable it can result in logging filling the disk, leading to an outage.
• To disable it revert the changes made to web.xml. Anything changed from "off" to "on" should be set back to "off" again.
• Save the web.xml file changes and restart the tomcat service.

Additional Information

Turn on DM debug for DataSourcePull and DataSourcePush. Ideally use the web URL method as follows:

• http://Portal:8481/dm/admin/config/logs?listing=described
• https://Portal:8482/dm/admin/config/logs?listing=described

Change DataSourcePull and DataSourcePush from INHERITED to DEBUG  and click the "Update" button at the top of the screen

If unable to access those URLs it can be enabled manually. It requires editing the (default path shown) /opt/CA/PerformanceCenter/DM/resources/log4j2.properties file. Doing so requires a restart of the Device Manager (caperfcenter_devicemanager) service. If that is restarted the Portal web UI Console service (caperfcenter_console) will also need to be restarted.

To do so open the file for editing and uncomment these lines as needed:

#logger.DataSourcePull.name = com.ca.im.portal.dm.productsync.DataSourcePull
#logger.DataSourcePull.level = DEBUG

#logger.DataSourcePush.name = com.ca.im.portal.dm.productsync.DataSourcePush
#logger.DataSourcePush.level = DEBUG

This will generate logging in the Portal machine /opt/CA/PerformanceCenter/DM/logs/DMService.log file.

Note: To disable Debug back to the original /opt/CA/PerformanceCenter/DM/resources/log4j2.properties file

On the Portal web page Manage Data Sources select Spectrum and Perform a Full or Incremental Resynchronization to generate the logging.