We have observed that the i_connecter is very unstable. We have observed that at times the oi_connector does not pass messages to AIOps and we have to restart the oi_connector and apm_bridge to resolve the issue.
qos_processor
Error in log:So we added the key:
database-update-queue-capacity and set it to 50000oi_connector loglevel
After setting the oi_connector loglevel back down from 5 (debug) to 1 (fatal error messages only) it allowed the queue to function more efficiently again and sent all of the messages and queued messages remained within a range of 0 and 2000 messages queued at any given time which is reasonable.
The QOS Message queue attached to oi_connector was then stable, and able to process the messages quickly and efficiently.
oi_connector probe axagateway.uimQos queue continues processing for several days or up to 2 weeks but when the purestorage probe is enabled, it starts to backup and cannot keep up.
apm_bridge:
Every time topology is created or updated it creates a file. A file is saved in the apm_bridge cache folder. If these files build up, while CPU/Memory usage slowly increases, the cache files can be deleted because they are only used temporarily
1. Deactivate the apm_bridge probe
2. Rename/delete the cache & store folder within apm_bridge. Cache folder will have a large number of 1 KB files in most deployments so deleting will take a very long time, so renaming the folder would work.
3. Activate the probe