Data not seen in ADA but seen in MTP
Article ID: 135933
Updated On:
Products
Issue/Introduction
When looking at ADA, I am not seeing data from my MTP, but data is seen directly in the MTP UI.
The last monitored date for the MTP on the Admin->Data Monitoring page of ADA is not current.
Environment
Resolution
1. SSH to the MTP in question and run the following command to see if ADA is successfully connecting to the MTP over port 8080 to pull the .dat files from the MTP's /opt/NetQoS/bin/SuperAgent/Web/datafiles directory over to the ADA's \CA\datafiles directory.
tail /var/log/httpd/access_log
If ADA is successfully pulling files its working you should see a 200 output like below
<ADA IP Address> - - [01/Dec/2012:15:10:23 -0500] "GET /getfile?do=getfile&file=v1291234200_5-6.dat HTTP/1.1" 200 16
<ADA IP Address>- - [01/Dec/2012:15:10:23 -0500] "GET /deletefile?do=deletefile&file=v1291234200_5-6.dat HTTP/1.1" 200
2. Check to see if there is a backlog of files on the MTP by running the following to get a file count, files should be moving in and out very quickly so you should only see a few files in this directory at a time, if the file count is low, skip to step 4. If there is a large backlog of files go to step 3.
cd /opt/NetQoS/bin/SuperAgent/Web/datafiles
ls | wc -l
3. If you are seeing a large backlog of files in this directory then it may be best to delete these files by running the command below on the MTP:
find . -name "*.dat" -print0 | xargs -0 rm
On the ADA Console server, restart the "CA ADA Master Batch" service once this completes and verify files are getting processed and moved over to the ADA \CA\datafiles directory and that they are getting processed from there.
4. If .dat files are getting moved over to the ADA \CA\datafiles folder but not getting processed from here, verify the the "CA ADA Inspector Service" is running, if its stopped start it, if its running restart it and see if files process.
5. If files are still not processing check the \CA\logs\Inspector_DatapumpModeMgr*.log file on the ADA Console for errors like below:
Error | Exception while processing file "D:\CA\Datafiles\R1565875800_4-4-0.dat": Lost connection to MySQL server during query
Error | Renaming to "D:\CA\Datafiles\R1565875800_4-4-0.dat.err"
Error | at Devart.Data.MySql.MySqlCommand.InternalExecute(CommandBehavior behavior, IDisposable stmt, Int32 startRecord, Int32 maxRecords)
6. If you see a message like above check the ADA super database to see the application and server count:
mysql super
select count(*) from applications;
select count(*) from servers;
This last one will give a count of how many system generated applications are in the database that are inactive.
select count(*) from applications where inactive=1 and usermodified='n';
7. Ideally you should have less then 1000 servers and 1000 applications, if you have far higher then that, the MTP is sending too much data to ADA and you need to limit the applications that get sent up to ADA.
The steps to do this are documented here, Disabling System-Defined Application Creation.
This will tell the MTP to only send up application/server pairs that are explicitly defined in ADA, so you will have to configure each application you wish to monitor in ADA.
8. Once this is set run through the steps below to cleanup the ADA database.
Stop the "CA ADA Inspector Service"
Delete all .dat files from the \CA\datafiles folder.
Run the following to delete the inactive applications:
mysql super
delete from applications where inactive=1 and usermodified='n';
Start up the "CA ADA Inspector Service" again and files should start processing again.
Feedback
