This NFMP is generating about 10K to 12K files an hour, however the device pack only seems to be processing about 8000 with a 15 minute poll interval, and we are seeing intermittent data on charts on some of the devices.
Also, I noticed the following message in one of the device pack logs:
Apr 27, 2021 11:30:23 AM, [[email protected]/Thread-7/168], com.torokina.tim.plugin.sftp.TimSftpDownloader recursiveDownload
INFO: Reached maxBatchSize = 1000, exiting Sftp Plugin
Here are the parameters are currently set in CA5620SamY1731-SAS.xml
TimSftpFunction: MaxDownloads=2000, and MaxBatchSize=1000
TimCopyFunction: MaxDownloads=1000
What is the correct way to modify the settings to be able to receive and process 12000 files per hour instead of 8000?
I bumped MaxBatchSize up to 3000 in CA5620SamY1731-SAS.xml, however, after doing that, I noticed that several devices were no long reporting data that had been before I made the change, so I rolled the change back.
Next I tried bumping up MaxDownloads in both the TimSftpFunction com.torokina.tim.plugin.sftp.TimSftpFunction and the com.torokina.tim.plugin.copy.TimCopyFunction functions, and got the same result (no data on some devices), so I again rolled the change back.
All supported DX NetOps CAMM CA5620SamY1731 device pack releases
To many small files at once for the CAMM code to process at once.
Engineering provided the following solution. It tells the code to combine the many small files into larger files to resolve the problem and keep up with the incoming rate of data.
The default section of the MergeCAMMXMLs.groovy script looks like:
cammxmls.each {
path->
path.each{
After editing the lines now look like:
cammxmls.each {
//path->
//path.each{