sfa3b.pl - "show flow active" analyser

book

Article ID: 168035

calendar_today

Updated On:

Products

XOS

Issue/Introduction

sfa3b.pl - "show flow active" analyserN/A

Cause

This allows processing of the text output from the XOS "show flow active" command into various useful forms.

This version of the "show flow analyser" (sfa3b.pl) is substantially updated from the previous version. It is for NPM-86xx-based system only (the older version can still be used for NPM-82xx)

Improvements include a comprehensive help system (try "sfa3b.pl --help") and the ability to modularise the output.

Resolution

 With very large input files, the default output options will potentially consume very large amounts of memory.
For example, a system with approximately 5 million flows requires up to 4GB of RAM.
This version allows you to specify that only the portions of interest (for example, the average TTL of flows) be processed and output.
By narrowing the focus of the script, the memory footprint can be substantially reduced.

Example

sfa3b.pl --file=medium.log --maxlines=100 --flowttl

This would process the file medium.log and only process it to produce the ordered set of Flow TTLs.
Additionally (although not required in this case) the output is limited to no more than 100 lines per section.

Note that the --pairs output (which is implicit in the default --all selector) is far and away the largest consumer of memory. It is best avoided unless required.

Modules Dependencies

The program has dependencies on a few Perl modules. Depending upon the Perl installation, some or all may well be already installed. If not, they are easily installed using the cpan command.
For example, start cpan, then type

         install Data::Dumper

The modules required (and the program will fail and indicate why if not installed) are:

Getopt::Long
Log::Log4perl
Pod::Usage
Data::Dumper

All are available for easy installation using the Perl cpan tool.

Web Access

In the near future it is planned to deploy this on a web-server. This will make use of the program much easier and simpler.

Until that is done, if there are any queries (for example, getting the module dependencies correct is a minor issue sometimes) then please get in touch and we can make the script available on a system for internal use.

Examples

   ./sfa3b.pl --file=large.log --maxlines=10 --portttl

+===================================================================================+
| Port TTL average- Source |
+===================================================================================+
| Source Port Frequency/Time |
+-----------------------------------------------------------------------------------+
| 80 6:58 |
| 443 7:22 |
| 5228 5:35 |
| 5223 5:37 |
| 3102 6:51 |
| 993 5:54 |
| 1863 8:06 |
| 0 0:18 |
| 3544 0:44 |
| 3103 6:50 |
+===================================================================================+
| Port TTL average - Dest. |
+===================================================================================+
| Dest Port Frequency/Time |
+-----------------------------------------------------------------------------------+
| 80 7:14 |
| 445 5:00 |
| 443 6:42 |
| 135 4:36 |
| 5228 5:31 |
| 6881 4:25 |
| 5223 5:30 |
| 3102 6:38 |
| 993 5:48 |
| 110 5:46 |
+===================================================================================+
+ TOTAL np6-style RECORDS PROCESSED: 4938813
+===================================================================================+



    ./sfa3b.pl --file=large.log --maxlines=10 --flowttl

+===================================================================================+
| Flow TTL (maximum) |
+===================================================================================+
| max ttl Frequency |
+-----------------------------------------------------------------------------------+
| 10:00 4139446 |
| 01:00 785564 |
| 00:30 12855 |
| 00:15 948 |
+===================================================================================+
+ TOTAL np6-style RECORDS PROCESSED: 4938813
+===================================================================================+




    ./sfa3b.pl --file=large.log --maxlines=10 --ipaddr --port

+===================================================================================+
| Source IP addresses |
+===================================================================================+
| source address Frequency |
+-----------------------------------------------------------------------------------+
| 209.85.229.188 98279 |
| 178.92.172.248 43670 |
| 69.63.190.10 40692 |
| 69.63.189.12 17581 |
| 62.233.116.31 11348 |
| 77.67.21.34 10897 |
| 209.85.227.118 10167 |
| 60.191.129.164 9966 |
| 69.63.190.18 9218 |
| 77.67.21.10 9076 |
+===================================================================================+
| Destination IP addresses |
+===================================================================================+
| destination address Frequency |
+-----------------------------------------------------------------------------------+
| 209.85.229.188 85599 |
| 69.63.190.10 39389 |
| 62.233.116.7 27586 |
| 69.63.189.12 15876 |
| 149.254.58.37 14762 |
| 77.67.21.34 9890 |
| 62.233.116.31 9364 |
| 209.85.227.118 8940 |
| 149.254.56.47 8439 |
| 77.67.21.10 8251 |
+===================================================================================+
| Source ports |
+===================================================================================+
| source port Frequency |
+-----------------------------------------------------------------------------------+
| 80 764011 |
| 443 103347 |
| 5228 98597 |
| 5223 30565 |
| 3102 26408 |
| 993 18017 |
| 1863 14697 |
| 0 12909 |
| 3544 6989 |
| 3103 6844 |
+===================================================================================+
| Destination ports |
+===================================================================================+
| destination port Frequency |
+-----------------------------------------------------------------------------------+
| 80 696080 |
| 445 571545 |
| 443 134884 |
| 135 106003 |
| 5228 85884 |
| 6881 37590 |
| 5223 28180 |
| 3102 24935 |
| 993 16106 |
| 110 15906 |
+===================================================================================+
+ TOTAL np6-style RECORDS PROCESSED: 4938813
+===================================================================================+

Workaround

N/A