Unable to Import SEPM log files into CASB-Audit with Error: Input file does not contain log messages required by Audit

Products

• CASB Securlet SAAS

Issue/Introduction

You are attempting to import log data from Symantec Endpoint Protection Manager (SEPM), via Web Upload in the CASB | Audit | Device logs console 

In the drop down menu you select the built-in 'Symantec Endpoint Protection Manager' option and select the zipped CSV data file

However the import eventually fails with 'Input file does not contain log messages required by Audit'

Cause

Data format mismatch:

Internally the SEPM data is marshalled by the JSON below: 

Note that the date format will change depending on if the tenant is US or EMEA region

{"logformat": "delimited", "delimiter": ",", "trim_tokens": "true", "bytes_val": "1", "src_index": "6", "dst_index": "8", "datetime_index": "3", "datetime_format": "MM/dd/yyyy HH:mm:ss", "action_index": "18", "action_blockedmatch": "Blocked", "user_index": "26", "url_index": "9"} 

 

Resolution

There are two possible approaches: 

 

1. Use Elastica Flex import option to change the configuration of the JSON parser to match the data

Instead of using the SEPM default drop down in the data import, you can select the 'Elastica Flex' option as documented in: Flex Universal Log Processor

You can then customise the Configuration JSON index fields to match the columns of your data that you already have. 

The advantage of this approach, is you can also insert additional fields in the JSON log format configuration as long as the added fields are in the Recognized log fields and you add the suffix  '_index' to them. 

So, for example if you wanted to add the Local Port from column 11 in your CSV data to your import job, you would add the section in green: 

{"logformat": "delimited", "delimiter": ",", "trim_tokens": "true", "bytes_val": "1", "src_index": "6", "dst_index": "8", "datetime_index": "3", "datetime_format": "yyyy/MM/dd HH:mm:ss", "action_index": "26", "action_blockedmatch": "Blocked", "user_index": "23", "url_index": "37","srcport_index": "11"} 

 

2. Change the format of the exported data to match the SEPM parser

This is the opposite approach to option 1

Change the exported data with text manipulation tool before importing into the CASB console. You will need to 

• Convert the date formats into those for your region in which your tenant is situated (if necessary)
• Change the order of the columns in the CSV file to match the data types anticipated in the JSON configuration fields above
• Note that only fields with the _index suffix in the JSON will be imported.