Intraday job is not calculating event normality

Products

• Information Centric Analytics

Issue/Introduction

When viewing the DLP Incidents by Individual Normality Rating dashboard in the Information Centric Analytics (ICA) console under Data Loss Prevention > DLP Overview, event normality scores do not appear to be updated outside of the nightly processing job.

Environment

Release : 6.x

Component : RiskFabric Intraday Processing

Cause

Although entity risk scoring only takes place during the nightly RiskFabric Processing job, event normality scoring is performed by both the nightly and intraday jobs. Possible reasons event normality scoring might not be updated intraday include:

1. The RiskFabric Intraday Processing job is disabled or not scheduled to run
2. One or more failures on the intraday job step Update LDW_Core

Resolution

To enable the RiskFabric Intraday Processing job, follow this procedure:

1. Open the Risk Fabric console
2. In the console, navigate to Admin > Integration > Job Status
3. Select the RiskFabric Intraday Processing job
4. Click the Status drop-down menu in the job list header and select Enable

The following queries can be used to investigate potential failures in event normality scoring:

SELECT   DISTINCT en.RFCreatedDate AS "ScoringDate",
         di.RFCreatedDate AS "StagingDate"
FROM     RiskFabric.dbo.LDW_EventNormality AS en WITH (NOLOCK)
JOIN     RiskFabric.dbo.LDW_DIMIncidents AS di WITH (NOLOCK)
ON       en.EventID = di.DIMIncidentID
WHERE    CAST(en.RFCreatedDate AS date) >= CAST(GETDATE()-7 AS date)
ORDER BY en.RFCreatedDate DESC;

SELECT  *
FROM    RiskFabric.dbo.Log_DataTransformation WITH (NOLOCK)
WHERE   StatusFlag = 'F'
AND     CAST(StartDate AS date) >= CAST(GETDATE()-7 AS date)
AND     (
            DestinationTableName = 'dbo.LDW_EventNormality' OR
            StoredProcedureName = 'spNormality_UpdateNormalityScores'
        )
ORDER BY 1 DESC;

SELECT      j.[name] AS "Job",
            s.step_id AS "StepID",
            s.step_name AS "StepName",
            h.[message] AS "Message",
            msdb.dbo.agent_datetime(run_date, run_time) AS "RunDateTime",
            ((h.run_duration/10000*3600 + (h.run_duration/100)%100*60 + h.run_duration%100 + 31 ) / 60) AS "Minutes"
FROM        msdb.dbo.sysjobs AS j
INNER JOIN  msdb.dbo.sysjobsteps AS s 
ON          j.job_id = s.job_id
INNER JOIN  msdb.dbo.sysjobhistory AS h WITH (NOLOCK)
ON          s.job_id = h.job_id 
AND         s.step_id = h.step_id 
AND         h.step_id <> 0
WHERE       j.[name] = 'RiskFabric Intraday Processing'
AND         s.step_id = 10
AND         msdb.dbo.agent_datetime(run_date, run_time) >= CAST(GETDATE()-7 AS date)
ORDER BY    h.instance_id DESC;