Our group_info queue in PROD is queued up and doesn't drain fast enough to keep up with the discovery messages. Also, after following the KB Article on how to disable the group_info queue, the queue simply reestablishes itself.
Release : 20.1
Component : UIM - OPERATOR CONSOLE
The group_info queue could not be removed. This issue should have been resolved as of 20.1 and there should be no group_info queue but the issue was reproducible with the same behaviour being seen in the lab (20.10).
The workaround was to DISABLE the queue under the Hub GUI Queues Tab, then empty the queue, then disconnect it. We left the queue disabled and then it disappeared from the Hub GUI Status window, otherwise if you deleted it, the hub would see it was missing and add the queue and reactivate it.
The probe_discovery queue was also backed up so we did some fine tuning to the udm_manager configuration, since it appears that after the last upgrade to 20.10, the settings were overwritten by the OOTB defaults.
Updated settings, see setup->datomic:
memory_index_max = 516m
memory_index_threshold = 64m
object_cache_max = 256m
heartbeat_interval_msec = 30000
After a few minutes we could see that the probe_discovery queue was processing messages efficiently with several thousand messages sent and only a few hundred in the queue, consistently.