UIM hub subscriber limits and how to monitor the count of subscribers
Article ID: 33649
Updated On:
Products
DX Unified Infrastructure Management (Nimsoft / UIM)
Unified Infrastructure Management for Mainframe
CA Unified Infrastructure Management SaaS (Nimsoft / UIM)
DX Unified Infrastructure Management (Nimsoft / UIM)
Issue/Introduction
The hub can only handle a maximum of 64 subscribers in a Windows environment due to Windows OS (winsock) limitations on the number of open sockets. As a result, you may see probes have their socket connections dropped or reset. This applies to all probes, both core and monitoring probes. The number of subscribers for Linux/Unix hubs is only limited by available resources.
Environment
- UIM (hub 7.x or higher)
Cause
- Guidance on alerting on the number of hub subscribers
Resolution
hub subscriber limits
Windows OS
The hub can only handle a maximum of 64 subscribers in a Windows environment due to Windows OS (winsock) limitations on the number of open sockets. So, if a hub has more than 64 subscribers, e.g., more than 64 queues, (which includes temp queues) then the hub will start to service the queues in a "round robin" fashion. This means one queue will get temporarily dropped while another one is picked up, then that one gets dropped and another one is picked up, etc.
This results in two undesirable effects.
One effect is that the hub's overall performance will be seriously degraded.
Another effect is that if the queue from a probe gets dropped and the probe notices this, it would cause the probe to restart itself in an attempt to re-establish its queue.
Windows only allows you to 'WaitForMultipleObjects' on 64 sockets, after which you have to split this information in two or more lists and WaitForMultipleIObjects on each of them. This causes an unacceptable delay in the message flow of messages on sockets not in the first list. Unix select() does not have this limitation, and so the limit is not relevant in Linux/UNIX.
Linux/UNIX OS
The number of subscribers for Linux/Unix hubs is only limited by available resources. Note that in Linux/UNIX, once again, temp queues count as subscribers. Note that the hub subscriber_max_threshold doesn't really matter for UNIX/Linux OS's as it does not suffer from the Windows limitation of WaitForMultipleObjects on 64 sockets. Though, on Linux it's been seen before that if it runs out of sockets and or open files, open files must be increased to mitigate this issue. You can increase the open file limit on the server. Normally open files limit is set to 1024. On Linux open files configuration can be checked with 'ulimit -a'
# ulimit -a // to check the system limits especially on open files. default is 1024 which hub could be exceeding
# lsof // get total number of open files while issue is occurring to make sure it's not the open file limit issue
Consult with a Linux admin if you need to increase the number of open files on a Linux system.
Checking the number of hub subscribers
Select the hub probe and hit Ctrl-p to open the hub probe utility and press the green arrow button to execute the callback list_subscribers to see the current total subscribers number at the bottom of the subscribers section of the resultant output - Note you MUST add 1 more subscriber since the list starts at 0.
Note that the list of subscribers starts at 0 and ends right before the section: postroute <TABLE START>
Windows OS
The hub can only handle a maximum of 64 subscribers in a Windows environment due to Windows OS (winsock) limitations on the number of open sockets. So, if a hub has more than 64 subscribers, e.g., more than 64 queues, (which includes temp queues) then the hub will start to service the queues in a "round robin" fashion. This means one queue will get temporarily dropped while another one is picked up, then that one gets dropped and another one is picked up, etc.
This results in two undesirable effects.
One effect is that the hub's overall performance will be seriously degraded.
Another effect is that if the queue from a probe gets dropped and the probe notices this, it would cause the probe to restart itself in an attempt to re-establish its queue.
Windows only allows you to 'WaitForMultipleObjects' on 64 sockets, after which you have to split this information in two or more lists and WaitForMultipleIObjects on each of them. This causes an unacceptable delay in the message flow of messages on sockets not in the first list. Unix select() does not have this limitation, and so the limit is not relevant in Linux/UNIX.
Linux/UNIX OS
The number of subscribers for Linux/Unix hubs is only limited by available resources. Note that in Linux/UNIX, once again, temp queues count as subscribers. Note that the hub subscriber_max_threshold doesn't really matter for UNIX/Linux OS's as it does not suffer from the Windows limitation of WaitForMultipleObjects on 64 sockets. Though, on Linux it's been seen before that if it runs out of sockets and or open files, open files must be increased to mitigate this issue. You can increase the open file limit on the server. Normally open files limit is set to 1024. On Linux open files configuration can be checked with 'ulimit -a'
# ulimit -a // to check the system limits especially on open files. default is 1024 which hub could be exceeding
# lsof // get total number of open files while issue is occurring to make sure it's not the open file limit issue
Consult with a Linux admin if you need to increase the number of open files on a Linux system.
Checking the number of hub subscribers
Select the hub probe and hit Ctrl-p to open the hub probe utility and press the green arrow button to execute the callback list_subscribers to see the current total subscribers number at the bottom of the subscribers section of the resultant output - Note you MUST add 1 more subscriber since the list starts at 0.
Note that the list of subscribers starts at 0 and ends right before the section: postroute <TABLE START>
***Alarming on the number of subscribers***
Using the hub Raw Configure you can add a NEW key and value to set the configuration parameter in the hub section to alarm when the number of subscribers reaches a specific value (see below).
subscriber_max_threshold
Specify the number of subscribers that must be reached to generate an alarm, e.g., 50.
subscriber_max_severity
Specify the severity of the max subscriber alarm above. This value is numeric.
Alarm severity values:
5 is Critical
4 is Major
3 is Minor
2 is Warning
1 is Informational
5 is Critical
4 is Major
3 is Minor
2 is Warning
1 is Informational
Additional Information
Normally the recommendation to resolve this issue is to off-load these to a "sibling" Unix hub. Unix systems do not have this limitation. You can also change the Get/Attach queues to POST and this will not count towards the subscriber limit. First step is to offload any monitoring probes or probes and their associated queues to another, e.g., secondary hub. Also note that UIM admin logins to IM count as a subscriber/temp subscribers so perhaps some admins can login to other hubs.
Feedback
Yes
No
Powered by
