Troubleshooting Flow Forensics Reports Errors

book

Article ID: 49074

calendar_today

Updated On:

Products

CA Network Flow Analysis (NetQos / NFA)

Issue/Introduction

Description:

Problem statement:

Flow forensics reports fail if two are run concurrently. When the reports fail, the system stops collecting data. The only way to recover the system is to reboot the console and the harvesters.

If at any point during this testing there is a failure. Please immediately alert CA support with the conditions of the failure and the specific steps that are taken.

Strategy:

  1. Collect Data

  2. Clean failed reports from the console

  3. Create test reports

  4. Run Baselines

  5. Test the console

  6. Test each harvester

  7. Test the entire system

  8. Analyze the data

Solution:

Collect Data:

  1. General System Data

    1. Write the IP addresses of each harvester and the console in this document

      • Harvester 1:

      • Harvester 2:

      • Console:

    2. Run a supportCIG on Harvester 1 (Physical), Harvester 2 (Virtual) and the Console,

    3. Get screen captures of all of the services running on each of the servers (Harvester1, Harvester2, and Console)

    4. Upload the files to the issue.

  2. Hardware Configuration:

    1. Console

      • Processor speed and # cores:

      • Memory Size:

      • Disk # Size and speed:

      • Raid configuration

    2. Harvester 1

      • Processor speed and # cores:

      • Memory Size:

      • Disk # Size and speed:

      • Raid configuration

    3. Harvester 2

      • Processor speed and # cores:

      • Memory Size:

      • Disk # Size and speed:

      • Raid configuration

  3. Determine the number of active interfaces monitored on each harvester

    1. Get the harvesterid's of each harvester by running

      Mysql -P3308 reporter

      Select distinct harvesterid from routers;

    2. Use each unique harvesterID in the query below separately, for example the three queries below are for harvesterID's 1 and 2

      Mysql -P3308 reporter
      select count(*) from agent_definitions where id in(select id from interfaces where enabled='y' and routerid in(select id from routers where harvesterid='1'));
      select count(*) from agent_definitions where id in(select id from interfaces where enabled='y' and routerid in(select id from routers where harvesterid='2'));

    3. Enter the data here:

      - <ip of harvester1>:
      - <ip of harvester2>:

  4. 4. Determine the total number of interfaces reporting to each harvester

    1. Run the NFAParser tool on each Harvester to gather this data.

    2. Enter the data here:

      • <ip of harvester1>:

      • <ip of harvester2>:

  5. 5. Determine the number of flows per minute seen by each harvester

    1. Run the NFAParser tool to get the Flows Per Minute total seen.

    2. Enter the data here:

      • <ip of harvester1>:

      • <ip of harvester2>:

Clean Failed Reports From The Console

  1. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/InitialFlowForensics.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  2. Upload InitialFlowForensics.csv to the issue from D:\InitialFlowForensics.csv'

  3. Open InitialFlowForensics.csv in Excel

  4. Note any reports with /N (null) in column i.

  5. These reports have never been run. Find them in the RA console interface and delete them.

  6. Note any reports with something other than /N in column K.

  7. These reports had errors and aborted during the run. Find them in the RA console and delete them.

  8. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/CleanFlowForensics.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  9. Upload CleanFlowForensics.csv to the issue from D:\CleanFlowForensics.csv

Create Test Reports

  1. Determine what interfaces to use

    1. Find two highly utilized interfaces on <ip harvester 1>

    2. Enter the Routers and interface names here:

      • <Router Name1> <Interface Name1>

      • <RouterName2><InterfaceName2>

    3. Find two highly utilized interfaces on <ip harvester 2>

    4. Enter the Routers and interface names here:

      • <Router Name3> <Interface Name3>

      • <RouterName4><InterfaceName4>

    5. Build the following FF reports with these characteristics

      • Name: FF Test Interface 1 Harvester 1

        1. Report Type: Conversation Sessions

        2. Start Date: Now-15 minutes

        3. End Date: Now

        4. Add Filters:

          1. RA: Interface equal <Interface1>

        5. Save, do not Run

      • Name: FF Test Interface 2 Harvester 1

        1. Report Type: Conversation Sessions

        2. Start Date: Now-15 minutes

        3. End Date: Now

        4. Add Filters:

          1. RA: Interface equal <Interface2>

        5. Save, do not Run

      • Name: FF Test Interface 3 Harvester 2

        1. Report Type: Conversation Sessions

        2. Start Date: Now-15 minutes

        3. End Date: Now

        4. Add Filters:

          1. RA: Interface equal <Interface3>

        5. Save, do not Run

      • Name: FF Test Interface 4 Harvester 2

        1. Report Type: Conversation Sessions

        2. Start Date: Now-15 minutes

        3. End Date: Now

        4. Add Filters:

          1. RA: Interface equal <Interface4>

        5. Save, do not Run

Run baseline reports

  1. Run Report: "FF Test Interface 1 Harvester 1"

  2. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface1Harvester1.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  3. Note anything odd that happens during running the report in the case.

  4. Upload FFTestInterface1Harvester1.csv to the issue from D:\FFTestInterface1Harvester1.csv

  5. Upload a pdf of the finished report to the issue

  6. Run Report: "FF Test Interface 2 Harvester 1"

  7. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface2Harvester1.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  8. Note anything odd that happens during running the report in the case.

  9. Upload FFTestInterface2Harvester1.csv to the issue from D:\FFTestInterface2Harvester1.csv

  10. Upload a pdf of the finished report to the issue

  11. Run Report: "FF Test Interface 3 Harvester 2"

  12. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface3Harvester2.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  13. Note anything odd that happens during running the report in the case.

  14. Upload FFTestInterface3Harvester2.csv to the issue from D:\FFTestInterface3Harvester2.csv

  15. Upload a pdf of the finished report to the issue.

  16. Run Report: "FF Test Interface 4 Harvester 2"

  17. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface4Harvester2.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  18. Note anything odd that happens during running the report in the case.

    Upload FFTestInterface4Harvester2.csv to the issue from D:\ FFTestInterface4Harvester2.csv

    Upload a pdf of the finished report to the issue.

Run two reports concurrently on different harvesters

  1. Run Report: "FF Test Interface 1 Harvester 1" and "FF Test Interface 3 Harvester 2"

  2. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface1and3.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  3. Note anything odd that happens during running the report in the case.

  4. Upload FFTestInterface1and3.csv to the issue from D:\FFTestInterface1and3.csv

Run two reports concurrently on the same harvester

  1. Run Report: "FF Test Interface 1 Harvester 1" and "FF Test Interface 2 Harvester 1"

  2. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface1and2Harvester1.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  3. Note anything odd that happens during running the report in the case.

  4. Upload FFTestInterface1and2Harvester1.csv to the issue from D:\FFTestInterface1and2Harvester1.csv

  5. Run Report: "FF Test Interface 3 Harvester 2" and "FF Test Interface 4 Harvester 2"

  6. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestInterface3and4Harvester2.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  7. Note anything odd that happens during running the report in the case.

  8. Upload FFTestInterface3and4Harvester2.csv to the issue from D:\FFTestInterface3and4Harvester2.csv

Run four reports concurrently

  1. Run Report: "FF Test Interface 1 Harvester 1","FF Test Interface 2 Harvester 1" ,"FF Test Interface 3 Harvester 2", and "FF Test Interface 4 Harvester 2

  2. Run the following commands:

    1. Mysql -P 3308

    2. Select * into outfile '/ FFTestallInterfaces.csv' fields terminated by ',' lines terminated by '\n' from reporter.flowforensicsreport_definitions;

  3. Note anything odd that happens during running the report in the case.

  4. Upload FFTestallInterfaces.csv to the issue from D:\FFTestallInterfaces.csv

Analyze the data

By analyzing which reports passed and which failed, the problem can be narrowed to a single piece of equipment or stage in the reporting process.

Environment

Release:
Component: NQRACO