CloudSOC JsonImporter fails processing with the error 'The value violates the MaxLength limit of this column'

book

Article ID: 211400

calendar_today

Updated On:

Products

Information Centric Analytics

Issue/Introduction

The Information Centric Analytics (ICA) JsonImporter fails to process data from Symantec CloudSOC and returns with the error "Cannot set column 'shared_with'. The value violates the MaxLength limit of this column". The following is seen in the importer logs:

JsonImporter.exe [1:ERROR] ApiClient.LogError() exception: Cannot set column 'shared_with'. The value violates the MaxLength limit of this column.
JsonImporter.exe [1:ERROR] ApiClient.LogError() System.ArgumentException: Cannot set column 'shared_with'. The value violates the MaxLength limit of this column.
   at System.Data.DataColumn.CheckMaxLength(DataRow dr)
   at System.Data.DataTable.RaiseRowChanging(DataRowChangeEventArgs args, DataRow eRow, DataRowAction eAction, Boolean fireEvent)
   at System.Data.DataTable.SetNewRecordWorker(DataRow row, Int32 proposedRecord, DataRowAction action, Boolean isInMerge, Boolean suppressEnsurePropertyChanged, Int32 position, Boolean fireEvent, Exception& deferredException)
   at System.Data.DataTable.InsertRow(DataRow row, Int64 proposedID, Int32 pos, Boolean fireEvent)
   at JsonImporterUtils.Data.SqlPersistor.<>c__DisplayClass13_5.<Stage>b__8(String x)
   at System.Collections.Generic.List`1.ForEach(Action`1 action)
   at JsonImporterUtils.Data.SqlPersistor.Stage(Endpoint endpoint, IEnumerable`1 data)
   at JsonImporterUtils.CloudSocApi.CloudSocSqlPersistor.Stage(Endpoint endpoint, IEnumerable`1 data)
   at JsonImporterUtils.CloudSocApi.ParseStageFactory.GenericParseAndStage(IPersistor persistor, String s, Endpoint endpoint)
   at JsonImporterUtils.ApiClient.GetResponse(Endpoint endpoint, IRequest request, Func`4 fParseAction)

Cause

The current value for the 'shared_with' column size is 128. If the data being inserted into the column are larger than the max column length, the job will fail to complete processing.

 

Environment

Version: 6.5.3, 6.5.4, 6.5.4 MP1

Component: JsonImporter

Resolution

As a workaround, manually increase the max lengths of the shared_with columns in the CloudSOCDW Investigate_shared_with staging tables, and increase the max length of the ApiAttributeName column in the EndpointAttributes table. This will allow longer string lengths (up to 256 characters) to be processed.

The following script will increase the maximum string length of both shared_with columns in the Investigate_shared_with tables, and the ApiAttributeName column in the EndpointAtrributes table in the CloudSOCDW database:

USE CloudSOCDW

ALTER TABLE stg_Investigate_shared_with
ALTER COLUMN shared_with NVARCHAR(256) NULL

ALTER TABLE Investigate_shared_with
ALTER COLUMN shared_with NVARCHAR(256) NULL

UPDATE EndpointAttributes
SET LENGTH = '256'
WHERE ApiAttributeName = 'shared_with'
;

A permanent fix will be included in the next release of ICA.