CloudSOC JsonImporter fails processing with the error 'The value violates the MaxLength limit of this column'

Products

• Information Centric Analytics

Issue/Introduction

The Information Centric Analytics (ICA) JsonImporter fails to process data from Symantec CloudSOC and returns with the error "Cannot set column 'shared_with'. The value violates the MaxLength limit of this column". The following is seen in the importer logs:

JsonImporter.exe [1:ERROR] ApiClient.LogError() exception: Cannot set column 'shared_with'. The value violates the MaxLength limit of this column.
JsonImporter.exe [1:ERROR] ApiClient.LogError() System.ArgumentException: Cannot set column 'shared_with'. The value violates the MaxLength limit of this column.
   at System.Data.DataColumn.CheckMaxLength(DataRow dr)
   at System.Data.DataTable.RaiseRowChanging(DataRowChangeEventArgs args, DataRow eRow, DataRowAction eAction, Boolean fireEvent)
   at System.Data.DataTable.SetNewRecordWorker(DataRow row, Int32 proposedRecord, DataRowAction action, Boolean isInMerge, Boolean suppressEnsurePropertyChanged, Int32 position, Boolean fireEvent, Exception& deferredException)
   at System.Data.DataTable.InsertRow(DataRow row, Int64 proposedID, Int32 pos, Boolean fireEvent)
   at JsonImporterUtils.Data.SqlPersistor.<>c__DisplayClass13_5.<Stage>b__8(String x)
   at System.Collections.Generic.List`1.ForEach(Action`1 action)
   at JsonImporterUtils.Data.SqlPersistor.Stage(Endpoint endpoint, IEnumerable`1 data)
   at JsonImporterUtils.CloudSocApi.CloudSocSqlPersistor.Stage(Endpoint endpoint, IEnumerable`1 data)
   at JsonImporterUtils.CloudSocApi.ParseStageFactory.GenericParseAndStage(IPersistor persistor, String s, Endpoint endpoint)
   at JsonImporterUtils.ApiClient.GetResponse(Endpoint endpoint, IRequest request, Func`4 fParseAction)

Environment

Version: 6.5.3, 6.5.4, 6.5.4 MP1

Component: JsonImporter

Cause

The current value for the 'shared_with' column size is 128. If the data being inserted into the column are larger than the max column length, the job will fail to complete processing.

Resolution

As a workaround, manually increase the max lengths of the shared_with columns in the CloudSOCDW Investigate_shared_with staging tables, and increase the max length of the ApiAttributeName column in the EndpointAttributes table. This will allow longer string lengths (up to 256 characters) to be processed.

The following script will increase the maximum string length of both shared_with columns in the Investigate_shared_with tables, and the ApiAttributeName column in the EndpointAtrributes table in the CloudSOCDW database:

USE CloudSOCDW;
GO

ALTER TABLE stg_Investigate_shared_with
ALTER COLUMN shared_with NVARCHAR(256) NULL;

ALTER TABLE Investigate_shared_with
ALTER COLUMN shared_with NVARCHAR(256) NULL;

UPDATE EndpointAttributes
SET  LENGTH = '256'
WHERE ApiAttributeName = 'shared_with';

A permanent fix will be included in the next release of ICA.