Data source query job scheduling and history
search cancel

Data source query job scheduling and history

book

Article ID: 171436

calendar_today

Updated On:

Products

Information Centric Analytics Data Loss Prevention Core Package

Issue/Introduction

Information Centric Analytics (ICA) pulls data from external data sources into ICA's pre-processing tables through the use of data source query jobs. These jobs can run hourly, daily, or weekly on a schedule defined by ICA administrators at the time data source queries are created, and can be adjusted later.

Environment

Version : 6.x

Component : Integration Wizard

Resolution

Each data source query job is a SQL Server Agent job that follows this naming syntax, where <n> is an integer corresponding to the data source query ID:

RiskFabric_IW_DataSourceQueryID_<n>

The SQL Server Agent job RiskFabric - Integration Wizard DataSource Query Manager is created the first time a data source query job is scheduled. This job queries the RiskFabric database to determine when individual data source query jobs should run.

Scheduling

To adjust the schedule of a data source query job, follow this procedure:

  1. In the ICA console, navigate to Admin > Integration > Data Sources
  2. From the dropdown menu labelled Choose Data Source, select User Defined
    This will present a list of all user-defined datasources that have been created in your ICA environment
  3. Locate the data source corresponding to the query you wish to schedule and expand it by clicking its + icon to reveal all queries that have been created for that data source
  4. Right-click the query for which you wish to modify its job schedule and select Edit Query
  5. Click the Watermarking / Scheduling tab
  6. Under the Scheduling heading, modify the job schedule as desired

NOTE: The nightly RiskFabric Processing job does not trigger individual data source query jobs and will only process records previously imported into the pre-processing tables by data source query jobs; consequently, data source query jobs must be scheduled to run prior to the nightly RiskFabric Processing job.

History

To view the details of a data source query's job status and last run history, the data source query's ID must first be identified. This ID is appended to the staging table name of data source queries when the staging table name is auto-created (e.g., Stg_15). If the staging table name was not auto-created, the ID can be identified by executing a query against the RiskFabric database. To query the ID, follow this procedure:

  1. Open SQL Server Management Studio (SSMS)
  2. Connect to the Database Engine hosting the RiskFabric database
  3. From the menu bar, select File > New > Query with Current Connection
    A new query editor window will open
  4. Copy the following query and paste it in the new query editor window: 
    USE RiskFabric;
    GO
    SELECT DataSourceQueryID,
           JobName
    FROM   dbo.IW_DataSourceQuery
    WHERE  DataSourceQueryName = '<query-name>';
  5. Modify the query by replacing <query-name> with the name of the data source query as displayed in the ICA console, or remove the WHERE clause from the query to return all data source query jobs and their query IDs
  6. Execute the query by pressing the F5 key, clicking the Execute button in the SQL Editor toolbar, or selecting Query > Execute from the menu bar
  7. Note the DataSourceQueryID and JobName returned under the Results tab

To view a data source query's job status and last run history, navigate in the ICA console to Admin > Integration > Job Status and lookup the job name corresponding to the JobName value returned by the query above.

To query a data source query's job history, including run statuses, statistics, and logging, follow this procedure:

  1. Open SSMS
  2. Connect to the Database Engine hosting the RiskFabric database
  3. From the menu bar, select File > New > Query with Current Connection
    A new query editor window will open
  4. Copy the following query and paste it in the new query editor window: 
    USE RiskFabric;
    GO
    SELECT   *
    FROM     dbo.vIW_DataSourceQueryHistory
    WHERE    DataSourceQueryID = <ID>
    ORDER BY StartDate DESC;
  5. Modify the query by replacing <ID> with the ID returned by the query in the preceding procedure
  6. Execute the query by pressing the F5 key, clicking the Execute button in the SQL Editor toolbar, or selecting Query > Execute from the menu bar

Data source query job statuses are indicated by the value set in the JobStatusFlag column. The JobStatusFlag values are:

  • R = Running
  • C = Completed
  • F = Failed

In the event of a job failure, error details will be logged in the ErrorDesc column. This, along with the details provided in the LogDescription column, can be used to trace, identify, and correct the cause of failure.

Additional Information

For additional information about adding data sources, creating data source queries, and scheduling data source query jobs, refer to the following sections of the Information Centric Analytics Administrator Guide:

About Symantec ICA Integration

https://techdocs.broadcom.com/us/en/symantec-security-software/information-security/information-centric-analytics/6-6/Integration-and-Solution-Accelerator-Guides/Integration-Guide/aboutIntegration.html

Using the Integration Wizard to Create a User-defined Data Source

https://techdocs.broadcom.com/us/en/symantec-security-software/information-security/information-centric-analytics/6-6/Integration-and-Solution-Accelerator-Guides/Integration-Guide/integrationBPs/userDefinedIWSource.html

Powered by Wolken Software