Managing Extractions and Reports from the CLI on Security Analytics

book

Article ID: 168911

calendar_today

Updated On:

Products

Security Analytics

Issue/Introduction

There may be large amounts of data in the extractions and reports from past work that is filling the hard disk or causing searches to take much longer than previously.  The large tables reduce the performance of the reports for the security analysts.  

Resolution

To see the current status of the tables in the the postgres database, you can run the following:   

scm db summary dsweb size - Look at the last few lines.  If the artifacts or meta_info tables are large, then extractions are consuming space.  If the report_items table is large, then reports are consuming space.  Reports and Extractions which are not Saved are typically deleted after 4-6 hours.

Extractions -
scm extractions summary - Shows a summary of extractions including whether they are  saved, the last access, the owner/creator, and the size in bytes.  Look for large extractions by size.  The files themselves are in /home/apache/artifacts/ID
scm extractions detail ID - Shows the metadata detail of an extraction, including the Path which would include the timespan for the files extracted.  A large timespan will produce
scm extractions delete ID - Allows the user to delete an extraction.
scm extractions artifact_list ID - Shows metadata for each artifacts.  Potentially a large list of files with size.

Reports -
scm reports summary - Shows the reports that are in the table.  
scm reports detail ID - Show the metadata on a report
scm report delete ID - Delete a given report.  Typically, has a large timespan.