NSX Network Detection and Response - Allow listing in NSX Lastline

Products

• VMware

Issue/Introduction

There are three main places that allow list can be set with varying degrees of functionality:

1. Network traffic on a Sensor
2. Message on a Sensor configured in a Mail processing mode
3. Portal User Interface

We will discuss each area below

1. Network Traffic on a Sensor

Overview

Customers on occasion would allow list certain activity from ever being picked up by the Sensor. We support the following forms of this:

• IP addresses. When allow listing an IP address, the Sensor will suppress Suricata events for the hosts in that allow list file (not just download activity).
• DNS domain names. When allow listing a domain name, blacklist hits against this domain will no longer occur, nor will HTTP downloads from those domains.
• MD5 hashes of files extracted from sniffed network traffic.
• Signature IDs (SIDs) of signatures matched by Suricata. The customer will not normally have insight into our signature IDs, so this is only for situations in which the customer would like to see specific signature hits disappear, and an SE looking up the corresponding SID.
  (Note: SIDs are NOT the detector IDs shown on the web portal)

Caveats

This feature is work-in-progress and a few things aren't finished yet:

• IP address allow listing currently doesn't support network prefixes, only exact IP addresses.
• For DNS domain names, allow listing doesn't currently cover all scenarios in which activity occurs on flows to destinations resulting from an earlier DNS A/AAAA record lookup. For example, allow listing example.com will not allow list any signature hit or FTP transfer in a flow to an IP address that results from a client's earlier DNS lookup of example.com. Note: There are two methods for allow listing domains, using the /etc/lastline/customer_whitelist_domains.txt option is the recommended approach. 
• llidsupload supports suffix-matching of domain names (i.e., foo.com will match a.foo.com and another.foo.com), whereas llpsv will match the domains exactly.
• The allow listing mechanisms currently apply to llpsv and llidsupload, not llmail.

Configuration

There are files in specific locations that you either create or update, followed by re-running the sensor configuration. If you have modified any of the below 4 mentioned files you can re-run the configuration by:

1. Logging into the Manager Web UI (On-Premise) or user.lastline.com (Hosted)
2. Navigate to the Admin section
3. Navigate to the Appliances section 
4. Click the Quick Links drop-down to the right of the sensor where you applied the allow list changes
5. Select Re-Trigger Configuration

This will reload the configuration (i.e., a puppet run) to update the necessary components.

The files to update are:

• /etc/lastline/customer_whitelist_domains.txt for DNS domain
• /etc/lastline/customer_whitelist_ips.txt for IP addresses
• /etc/lastline/customer_whitelist_filehashes.txt for MD5s
• /etc/lastline/customer_whitelist_signatures.txt for signature IDs

Only list one entry per line in the files:

# separate line comment
# I trust the following domain
www.google.com

2.  Messages on a Sensor configured in a Mail processing mode

To allow list domains that are part of the emails, the following file can be edited to add the domain:

• /etc/lastline/customer_whitelist_domains.txt for DNS domain

• Example to allow list domains for llmail:

  ​web.abc.com

  
  Following is the corresponding entry to be added in /etc/lastline/customer_whitelist_domains.txt and followed by llmail restart:

  ​.*web\.abc\.com

   

There are 6 different variables that can be set in the "/etc/appliance-config/override.yaml" file on the Sensor to configure your allow list for Mail processing.

       a. llmail::url_whitelist
       b. llmail::attachment_md5_whitelist
       c. lmail::attachment_name_whitelist
       d. llmail::recipient_address_whitelist
       e. llmail::sender_address_whitelist
       f. llmail::subject_whitelist

The value assigned to these variables should be the full path to a text file you create that contains either a list of MD5 checksums for object allow listing or a list of Python-compatible regular expressions for allow listing everything else.

Examples: 

a. llmail::url_whitelist: <path to whitelist file>
This whitelist file must contain a Python-compatible regular expression per line. If a URL in an email matches any of these regular expressions, the URL is not analyzed.

Create your url whitelist file in a convenient location: 

"/home/lastline/company_url_whitelist.txt".

This file would have lines that look like this:
 .*\.acme\.some-domain\.com

b. llmail::attachment_md5_whitelist: <path to whitelist file>
The allow list file must contain a md5 per line. If the md5 of an attachment matches any of these lines, the attachment is not analyzed.

Create your attachment allow list file in a convenient location:  "/home/lastline/company_attachment_md5_whitelist.txt".

This file would have lines that look like this:

e7b9653f64b8cd895bc8e241e059787a
4101AC7D217A285D16C601AFD4A32BEB
663a7191bea8bb10dbbe67e60629b487
0c9fa877f05e3a7a911eaa813f8ee805
44d88612fea8a8f36de82e1278abb02f 

c. llmail::attachment_name_whitelist: <path to whitelist file>
The allow list file must contain a Python-compatible regular expression per line. If the filename of an attachment matches any of these regular expressions, the attachment is not analyzed.

d. llmail::recipient_address_whitelist: <path to whitelist file>
The allow list file must contain a Python-compatible regular expression per line. If all the recipient addresses of an email match any of these regular expressions, no attachment or URL in the email gets analyzed.

e. llmail::sender_address_whitelist: <path to whitelist file>
The allow list file must contain a Python-compatible regular expression per line. If the sender address of an email matches any of these regular expressions, no attachment or URL in the email gets analyzed.

f. llmail::subject_whitelist: <path to whitelist file>
The allow list file must contain a Python-compatible regular expression per line. If the subject of an email matches any of these regular expressions, no attachment or URL in the email gets analyzed.

Then, add lines for each allow list file you created into /etc/appliance-config/override.yaml:

llmail::attachment_md5_whitelist: /home/lastline/company_attachment_md5_whitelist.txt

 

Notes:
- The files are not on the system by default and need to be manually created.
- After modifying the file /etc/appliance-config/override.yaml, you need to run the command lastline_apply_config in a shell, or you can click the "retrigger configuration" button from the web portal in order for these variables to take effect on the Sensor.
- When modifying ONLY the whitelist files and no changes were made to the file /etc/appliance-config/override.yaml a manual refresh of the llmail service is needed, to do so run the following command in a shell: service-lastline llmail-daemon restart

 

3. Portal User Interface

The allow listing features can be found under Network > Network Settings:

There are two allow list features available. The recommended one is "Alert management rules". If allow listing certain client IP addresses is required, click "Silenced host IPs" to do so.

 

3.1 Alert suppression rules

(NOTE: this feature is available since hosted 2019.5 and on-premise 8.4)

The most recommended way to setup allow list in portal is to use "Alert management rules". By creating flexible rules, user can either demote the uninterested events into INFO level, or to delete (suppress) events. This feature is under Network > Network Settings:

There are two kinds of rule scopes, "customer-scoped" or "license scoped". This is because a single customer might have multiple licenses. If a license-scoped rule has the same name as a customer-scoped rule, the license-scoped rule has precedence for the specific license. The customer-scoped rule will be marked to indicate that it has been overridden.

The matching expression of the rule is a number of filters that are matched against events. The expression may be truncated if it is too long. Expand the row to display the full content of the rule by clicking the + icon (or anywhere on the entry row).

The same rules can also be managed from "Event Profile > Manage Alert and from Event Summary > Manage":

For more detailed information on how to build the rule, please refer to "Alert suppression syntax".

3.2 Hosts Allow list

To allow list based on the client host IP address(es), simply choose the right license key and Sensor name. Name a label, enter start and end IP range (use the same IP if this is to apply for a single host), click "whitelist range" to take effect. 

For more information on allow listing host(s) please refer to Silenced host IPs tab and Host labels tab. 

 

**Below is for on-premise release prior to 8.4:

There are 2 places in the Portal User Interface where "allow listing" is configured. In this case, "allow listing" refers to the ability to create persistent display filters that will be retained across login sessions as a means of retaining the view filters of your choice.

The two places are in the portal are the 

 

tab located on the top of the Dashboard, Console, Events & Downloads tabs. When selected, you are presented with the options of whitelisting either Hosts or IP ranges

3.1 Label/Whitelist Hosts

If you select Label/Whitelist Hosts you are prompted to select a particular sensor

Select the Sensor you are interested in and then click Apply. The next screen

Allows you to create a whitelist entry for the host you are interested in by providing a label and either a hostname or ip address. Once you have created an entry it will look like this

Notice that you can turn the whitelisting on or off by checking the "Whitelist host" box when creating or editing the whitelist entry.

3.2 Whitelist IP ranges

If you select Whitelist IP ranges, you are again prompted to select the sensor for which you would like this whitelist to apply to. Then you are presented with

which allows you to create a Whitelist IP range and give it a label. Once you have done this it will look like,

Additional Information

Note: This article is applicable to the standalone NSX Network Detection and Response product (formerly Lastline) and is not intended to be applied to the NSX NDR feature of NSX-T.