CSV record limits

Products

• Information Centric Analytics

Issue/Introduction

Is there a limit to the number of records that can be imported into Information Centric Analytics (ICA) from a File System Integration Wizard (IW) data source? What is the maximum number of records a comma-separated value (CSV) file can contain and still be imported into ICA?

Environment

Version : 6.x

Component : Integration Wizard

Resolution

ICA does not have an upper limit to the number of records it can import from a CSV. The IW reads CSV files in batches of 1,048,576 rows, staging each batch in a temporary table in the RiskFabric database, before processing the records into staging tables for later processing into ICA's core tables. As is the case with all data sources, the volume of data in relation to the resources allocated to the SQL Server host will determine the performance of the import.