Why are the *.tbn files in my Harvester's ReaperInput directory are processing very slowly?

Products

• CA Network Flow Analysis (NetQos / NFA)

Issue/Introduction

Why are the *.tbn files in my Harvester's ReaperInput directory are processing very slowly?

Environment

Release : Any NFA Version

Component : NQRASH - NETQOS RA SUPER HARVESTER

Resolution

 

\CA\NFA\Netflow\datafiles\ReaperInput\*.tbn files are incoming datafiles that are waiting to be processed by the reaper to parse out 1 minute and 15 minute resolution data.

 

When the Harvester is scaled properly the *.tbn files should get processed very quickly.

 

There are two main factors that could cause these files to get processed slowly.

 

1. The disk speed allocated to the server is too slow.  The reaper service utilizes a lot I/O when processing files. 

 

2. There are too many flows per minute and/or interfaces sending flows to this Harvester.

 

The recommended flows per minute and disk speed recommendations can be found in the link below:

System Recommendations and Requirements (broadcom.com)

 

The best way to determine the raw flows per minute is by looking at the Flow Statistics page under "Administration."

 

 

Alternatively, another way would be to look at the size of the *.tbn files:

RDP into the Harvester and navigate to the <Install Directory>\CA\NFA\Netflow\datafiles\ReaperInput directory and check the size of the *.tbn files in this directory.

If you take the size of the file in bytes and divide by 64 and this will give a rough estimate of the flows per minute on the Harvester.

For example if you have a 1GB tbn file which is 1073741824 bytes divided by 64 equals roughly 16,777,216 flows per minute.

In that scenario your Reaper Service will have a difficult time keeping up with the incoming data and could lead to your drive filling up and you will see a delay in data when looking at report data.

 

In order to resolve this, you can do one of the following.

 

1. Redirect flows from some of your routers with the most flows per minute to another Harvester.

You can determine which routers have the highest flows per minute by using the Admin->Flow Statistics page in NFA and expanding a specific harvester in the table at the bottom, which will show all routers for that Harvester. 

You can then sort by Max Flows per Minute.

 

2. Disable interfaces on the routers with he heavier flows per minute which will reduce the size of the tbn files.

 

3. Change the router configuration to use a sampling rate which can reduce the flows per minute.

 

4. Limit the number of interfaces enabled to send Netflow within the router configuration.