processFileLines gel tag as per:
It’s supposed to process each line in turn in order to keep memory low, by not reading in the whole file into memory.
In reality, due to failing to reset some array of each line’s values, it ends up reading the whole file into memory anyway, but makes it harder to work with than before, and just moves where the accumulation takes place.
Attached a zip with a gel script, a .csv file, and an example of its output in a text file.
1. Copy the GEL file attached
2. Go to a 15.6 system
3. Connect to Command Prompt and run from /bin directory:
This should be enough to reproduce the issue, if the csv is in the same location too.
Expected Results: Output the data without duplication, each line should be new without old data
Actual Results: Old data is pulled into the results and the tag is unusable as it iterates over the old rows
See attached csv and output
NOTE: It affects scripts executed by process steps too, it’s just quicker to reproduce the issue from the command prompt.
Release: CODSFF99100-15.6-PPM SAAS-Full Function User
This is reported as DE48306, fixed in 15.6.1
It’s similar to DE37803 reported previously for reading of multiple files (fixed in 15.4) except that this affects the reading of multiple lines within a single file. Reference: