RiskFabric Processing and RiskFabric Intraday Processing job step descriptions

Products

• Information Centric Analytics

Issue/Introduction

Refer to the Resolution section of this article for a basic high-level description of each individual step that's run by the nightly RiskFabric Processing job and the RiskFabric Intraday Processing job.

Resolution

RiskFabric Processing (6.6.x)

Job Step ID	Job Step Name	Job Step Description
1	Check for Upgrade or previous nightly jobs running	Checks for any currently running nightly or intraday RiskFabric processing jobs, including upgrade and staging jobs. If this check evaluates to yes, the current job instance terminates.
2	Pre Job Processing	Captures snapshots of user and person entities for comparison with changes made during core data processing.
3	Update Organizations table	Executes the stored procedure spLDW_UpdateCustomOrganizations, which can be customized to modify data specific to organizations. By default, this stored procedure does nothing.
4	Run Staging Procedures	Imports data from integrated data sources into staging tables within the RiskFabric relational database.
5	Run User Loading Procedures	Processes user entity data from staging tables into normalized pre-processing tables and from there into Risk Fabric’s core (LDW) tables.
6	Run Loading Procedures	Processes non-user entity data from staging tables into normalized pre-processing tables and from there into Risk Fabric’s core (LDW) tables.
7	CustomUserData	Executes the stored procedure spLDW_UpdateCustomUserData, which can be customized to modify data specific to user entity records. By default, this stored procedure does nothing.
8	CustomComputerEndpointData	Executes the stored procedure spLDW_UpdateCustomComputerEndpointData, which can be customized to modify data specific to computer endpoint entity records. By default, this stored procedure does nothing.
9	Reconcile Organizations and Countries	Enriches user entity records with organization and country identifiers as defined in your instance of Risk Fabric.
10	Update Agent Applicability	Executes a stored procedure to update the statistics used on the Agent Coverage dashboard. This dashboard is meant to track the health of the various agents in the organization including indicating computer endpoints detected that have an operating system and device type indicating it should have a particular agent but does not. This is useful in determining systems that may be missing the necessary security agents to be properly protected.
11	Update LDW_Core	This step runs a number of background processing functions including index management, table partitioning and pruning, and data normalization and cleansing, among others.
12	Custom Policy Data	Executes the stored procedure spLDW_UpdateCustomerPolicies, which can be customized to modify data specific to policies. By default, this stored procedure does nothing.
13	Update Policy Order	Updates the display order of Risk Fabric policies to sort alphabetically.
14	CustomDIMData	Executes the stored procedure spLDW_UpdateCustomDIMData, which can be customized to modify data specific to data in motion (DIM) incidents. By default, this stored procedure does nothing.
15	CustomComplianceData	Executes the stored procedure spLDW_UpdateCustomComplianceData, which can be customized to modify data specific to vulnerability compliance data. By  default, this stored procedure does nothing.
16	Risk Fabric cube processing	Processes data from core tables into the RiskFabric OLAP cube.
17	Process Risk Models	Generates risk model instances.
18	Process Collections and Scenarios and Risk Model Measure Groups	This step reprocesses the measures and dimensions affected by the Process Risk Models step to synchronize the data between the relational database and the OLAP cube.
19	Calculate Risk Scores	Calculates entity risk scores from risk vectors.
20	Process Risk Vectors Measure Group	This step reprocesses the measures and dimensions affected by the Calculate Risk Scores step to synchronize the data between the relational database and the  OLAP cube.
21	Delete Todays Metrics	The Metric feature of the product takes a daily snapshot of any defined metrics. This step deletes today's collected data which were collected by the intraday job or in the event the nightly job had been run manually since its last scheduled run.
22	Metric Collector	This step takes a snapshot of defined metrics and stores them for use in the console.
23	Update DB Utility Versions	Identifies the current versions of Risk Fabric’s database utilities and records these values in the Versions table.
24	Update Data Refresh Date	Records a timestamp of the completion of data processing by the current instance of the RiskFabric Processing job.
25	Send License Expiration Notifications	Sends license expiration notifications by e-mail to the License Expiration Recipient Address value configured in the Risk Fabric console’s general portal settings.
26	Send Scenario Step Up Emails	Sends event scenario step-up notifications by e-mail to named recipients as configured per event scenario.
27	Send Email Notifications	There are several features inside the Risk Fabric console that can be configured to send notifications. This step collects and sends these e-mail notifications.
28	Purge Activity Log Data	Purges records from the ActivityLog table that exceed the Activity Log retention days value configured in the Risk Fabric console’s general portal settings.

 

RiskFabric Processing (6.7)

Job Step ID	Job Step Name	Job Step Description
4	Process Symantec DLP Purge Requests	Executes the stored procedure spDBA_ProcessSymantecDLPPurgeRequest to purge incidents from the RiskFabric relational database by LinkedServer /  PurgeDate based on purge requests configured in the Risk Fabric console.
5	Process Symantec DLP Staging Reingest Requests	Executes the stored procedure spDBA ProcessSymantecDLPStagingReingestRequest to re-import previously skipped DIM incidents for a given policy  (effectively ignoring the watermark) based on reingestion requests configured in the Risk Fabric console.

 

RiskFabric Intraday Processing (6.x)

Job Step ID	Job Step Name	Job Step Description
1	Check for Upgrade or nightly jobs running	Checks for any currently running nightly or intraday RiskFabric processing jobs, including upgrade and staging jobs. If this check evaluates to yes, the current job instance terminates.
2	Pre Job Processing	Captures snapshots of user and person entities for comparison with changes made during core data processing.
3	Update Organizations table	Executes the stored procedure spLDW_UpdateCustomOrganizations, which can be customized to modify data specific to organizations. By default, this stored  procedure does nothing.
4	Run Staging Procedures	Imports data from integrated data sources into staging tables within the RiskFabric relational database.
5	Run User Loading Procedures	Processes user entity data from staging tables into normalized pre-processing tables and from there into Risk Fabric’s core (LDW) tables.
6	Run Loading Procedures	Processes non-user entity data from staging tables into normalized pre-processing tables and from there into Risk Fabric’s core (LDW) tables.
7	CustomUserData	Executes the stored procedure spLDW_UpdateCustomUserData, which can be customized to modify data specific to user entity records. By default, this stored procedure does nothing.
8	CustomComputerEndpointData	Executes the stored procedure spLDW_UpdateCustomComputerEndpointData, which can be customized to modify data specific to computer endpoint entity records. By default, this stored procedure does nothing.
9	Reconcile Organizations and Countries	Enriches user entity records with organization and country identifiers as defined in your instance of Risk Fabric.
10	Update LDW_Core	Much like the step of the same name in the nightly job, but runs a smaller subset of background tasks to decrease the amount of run time.
11	Custom Policy Data	Executes the stored procedure spLDW_UpdateCustomerPolicies, which can be customized to modify data specific to policies. By default, this stored procedure does nothing.
12	CustomDIMData	Executes the stored procedure spLDW_UpdateCustomDIMData, which can be customized to modify data specific to Data In Motion (DIM) incidents. By default, this stored procedure does nothing.
13	Process DIM Measures and Dimensions	As new data are ingested from Symantec DLP as a part of the intraday job, this step updates the related OLAP cube objects to keep the Analyzer and the source data in sync.
14	Process Event Scenarios	Generates event scenario instances for Data In Motion (DIM) incidents.
15	Process Event Scenario Measures and Dimensions	Updates the OLAP cube’s measures and dimensions affected by the above processing step to keep the Analyzer and source data in sync.
16	Update Data Refresh Date	Records a timestamp of the completion of data processing by the current instance of the RiskFabric Intraday Processing job.