Altering staging table data types and column sizes

Products

• Information Centric Analytics

Issue/Introduction

Can the data types or column lengths of a data source query's staging table be altered to address failures during staging caused by changes in source system data?

Environment

Release : 6.x

Component : Integration Wizard

Resolution

Contact Broadcom Support for assistance with determining the correct approach to handling changes in source system data types and lengths.

Additional Information

Information Centric Analytics (ICA) offers users the ability to integrate additional data sources beyond those for which integration packs are provided. These integrations are created via the Integration Wizard (IW) and are configured in three components:

1. Data Source
2. Data Source Query
3. Import Rule Mapping

When configuring the data source query for an integration, the user is prompted to provide a destination table in which data returned by the query will be stored, or staged. This table is created upon saving the query and its column sizes and data types are determined based on a sample result set of rows returned by the query. Because this is a sample, it's possible the string lengths returned are not representative of the maximum lengths stored in the data source.

NOTE: Source data need to be transformed to fit the data types and column length limits of the pre-processing columns to which they are mapped. These transformations can be applied in the data source query or via an import rule mapping formula. Altering staging columns to accommodate data types and string lengths that exceed or are incompatible with the destination pre-processing columns will necessitate the use of import rule mapping formulas.