How to bypass the index to download a PCAP file from the "merge" path on Security Analytics

book

Article ID: 168654

calendar_today

Updated On:

Products

Security Analytics

Issue/Introduction

When you download a PCAP file from a Security Analytics server, it uses what is called the "gauge path" which uses the meta data (index data) to locate the raw packets needed for the PCAP download.  If there is ever a time where the index data is questionable or is missing information, you can bypass the index and pull the PCAP directly from the raw packet data.  

NOTE:  This method of pulling a PCAP from the "merge" path can be quite a bit slower and is limited to BPF filter syntax only, which uses packet-based criteria (e.g. ip address, port, protocol).  Information that is flow-based such as DNS query or URL or social persona is not allowed in a BPF filter.  This method of downloading a PCAP ignores flow boundaries.

Resolution

You can generate this sort of PCAP file from the GUI or from the command line.  The GUI method will generate an aggregated PCAP file from all capture ports.  If you need to grab data from a specific capture interface, you must use the CLI method.  

NOTE:  If you are running Solera OS version 6.x or earlier, you must use the CLI method.

GUI method
  1. Log in to the GUI and go to the Analyze Summary page.
  2. Specify the timeframe you are interested in.  DO NOT put any additional filter criteria in the filter bar.  It will be ignored.
  3. Select Actions > Download PCAP.
  4. For the Type, select PCAP without Filters.
  5. Under Filter, select Create New Filter.
  6. Give the Filter a name and then in the BPF Expression field, enter the BPF filter you would like to apply to this PCAP.  Examples of BPF filters can be found in the Security Analytics Administration Guide or can be searched for on the Internet.  The most common example is a single IP address, which would be:   (host xx.xx.xx.xx)   This would only find packets where the source or the destination IP matches xx.xx.xx.xx.
  7. Click to Download the PCAP file.
CLI Method
  1. Log in as root via an SSH session.
  2. cd to the /pfs/merge/ directory
  3. Create a new directory in /pfs/merge using the following syntax. 
interface-MM.DD.YYYY.HH.MM.SS:d-MM.DD.YYYY.HH.MM.SS:d
  1. The first set of dates is the start date/time and the second set is the end date/time.  For example, if you wanted to download all traffic captured on March 21st, 2013 between 10:38:50 and 10:38:59 from capture interface eth2, you would execute the following command:
mkdir eth2-03.21.2013.10.38.50:d-03.21.2013.10.38.59:d
 
If you wanted to download a PCAP for the same timeframe, but from 2 capture interfaces (eth2 and eth3), you would use the following command:

mkdir eth2:eth3-03.21.2013.10.38.50:d-03.21.2013.10.38.59:d  
  1. cd into newly created directory and if you run ls, you'll notice a data.pcap file.  If the data.pcap file does not exist, check the syntax on the directory you created and try again.  This is an unfiltered PCAP file. 
  2. In order to download the PCAP file, you must copy the file to the /home directory first before you SCP the PCAP file off the server.  This is a required step due to the nature of the virtual filesystem that Security Analytics uses.
cp data.pcap /home/
  1. If you want to further filter down the PCAP file using a BPF filter, create a new text file in the directory where the data.pcap file exists.  The easiest way to do this is to use this command:
echo "(BPF expression)" > filter.txt

For example, if you wanted to only download a PCAP file that only contained IP address 10.10.10.10, you would enter:  echo "(host 10.10.10.10)" > filter.txt
  1. Apply the filter to the data.pcap file using the dsfilter command as follows:
dsfilter -m -f <filter_file> <source.pcap>

In our example above, you would enter this:

dsfilter -m -f filter.txt data.pcap
  1. This creates a new PCAP file called <name_of_filter_file>.pcap.  Copy this filtered PCAP file to the /home directory as explained above prior to SCP'ing the file off the appliance.

NOTE:  If interfaces are aggregated, you need to use agg0, agg1, etc. in place of eth2, eth3, etc.