ADA stopped collecting data and files are not processing in the \CA\datafiles folder

Products

• CA Application Delivery Analysis MTP (NetQoS / ADA)

Issue/Introduction

ADA stopped collecting data and files are not processing in the \CA\datafiles folder.

The Inspector_DatapumpModeMgr_*.log that there are errors like below:

"D:\CA\Datafiles\R1521090600_7-2-16.dat": Lost connection to MySQL server during query"

Environment

Release:
Component: NQSAMC

Cause

This is usually caused by an over loaded ADA server.   It is recommend that ADA have no more then 1000 Servers and Applications in the database, once a system goes over this level, it will be difficult to recover from.

These requirements are documented below:
https://docops.ca.com/ca-application-delivery-analysis/11-0-2/en/getting-started/monitoring-with-ca-application-delivery-analysis/monitoring-tcp-sessions/limitations-of-applications-and-servers-on-ada-console

You can check the super.servers and super.applications tables in mysql to get a count of how many servers and applications you have. If they are both much over 1000, this is the likely cause of the issue.

mysql super
select count(*) from servers;
select count(*) from applications;

Resolution

1. Ensure that the "Catch All" server subnet that is set by default is deleted as this will usually bring in too many servers.

2. On each Collector/MTP set the "force positive config" setting as documented in the link below to ensure that ADA only created User Modified Applications, this will prevent System Defined Applications from being created and causing the database issue.
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/application-delivery-analysis/11-1/managing/console/managing-applications/system-defined-applications/disabling-system-defined-application-creation.html

3. If there are too many server and applications in the system already, you may have to uninstall ADA and reinstall ADA with a fresh database. 

**Note**
You may want to export your list of servers and make note of your user defined Application Definitions before uninstalling.
If you have setup HTTPS you may want to back up the files and directories listed in HTTPS setup instructions below:
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/application-delivery-analysis/11-1/installing/post-installation-configuration/enable-https-for-the-ca-application-delivery-analysis-web-interface.html

4. When uninstalling follow the steps in https://knowledge.broadcom.com/external/article?articleId=75046

5. After reinstalling, and before adding any Collectors or MTPS delete the 0.0.0.0 Server Subnet from the Admin->Data Monitoring->Servers, Server Subnets page.

6. Add the collectors back and verify that files are processing.

7. If needed setup HTTPS if needed by following the steps: 
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/application-delivery-analysis/11-1/installing/post-installation-configuration/enable-https-for-the-ca-application-delivery-analysis-web-interface.html