Primary hub slows down / become unusable. login [NimBUS] - failed for user user=example as password has expired

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The DX UIM server (primary hub) became very slow and unmanageable. When connecting to a secondary Hub in IM from an other hub, the primary hub shows up but unresponsive. Admin console logs in but extremely slow. OC also impossible to log in because of slowness of server. 

Nimsoft service restart does not help. 

On hub.log analysis it is visible a recurrent log failure: (very often) 
 

Oct  8 10:01:46:454 [11584] 3 hub: login [NimBUS] - failed for user=<nimbus_user> ip=###.##.##.# as password has expired


and also: 

Oct  8 10:01:46:053 [11584] 0 hub: (nim_ldap_get_connection): LDAP server spec '<ldap_server> failed (secure=0)
hub: login [LDAP] - auth (ldap_simple_bind_s) failed: 'Invalid Credentials' (49)

Environment

DX UIM 23.4.4 (CU4) and later

Cause

• Password expired for nimbus user can cause this issue If DX UIM is integrated externally via REST API's (Eg. an authorized user - Nimbus user - pulls out alarms every 2 minutes by executing a remote Rest call to uimapi) and LDAP is enabled on the primary hub. 
  
  
• Recurrent and repeated failed login attempts in short amount of time when LDAP is enabled can overload the primary hub and turn it unusable. Turning LDAP off alone removes the issue, as the repeated failed log in attempts alone do not cause this issue. What causes the issue is the combination of repeated expirted password failed logging attempts while also having LDAP Enabled in the DX UIM server / primary hub. 
  
  
• When LDAP is enabled, at every failed attempt of any user, in this case the REST API user, the hub will always try to find that user in the Active Directory connected via LDAP. In large Active Directory environments this can cause resources overload on the hub to the point it becomes unreachable or too slow to use. 
  
  
• In normal conditions, when the REST user can log in correctly, the recurrent (eg. 2 minutes) authentications to the HUB, even if they are sent at short intervals, they are not all authenticating to the hub, but at the first authorized "token" allowed by the hub it will remain logged in for a longer period (Eg. 20 minutes - the availability time of the token) so between the connections to the LDAP server, there is enough time for the hub to work normally. 
  
  
  
  NOTE: 
  
  
  • Since DX UIM 23.4.4 (CU4) DX UIM enables by default new Password Policies: The password (for users) expires after X days (configurable, for more information, see:
    Customize the Password Expiration and Notification Interval)
    
    
  • By default, the user is notified on-screen 10 days before the expiry of the password. However, this warning only appears once you log in to IM / Admin Console or OC. So if a rest user logs programmatically via script, won't get this warnings and will cause this issue. 

Resolution

• If running DX UIM with some external integration that uses a Nimbus user to extract UIM data via REST API / uimapi / webservices_rest, and at the same time you have a large LDAP integrated Active Director, update the Nimbus User password and review other similar users in time before password expires. -- >  Customize the Password Expiration and Notification Interval)
• By attempting log-in to IM to OC or Admin console with the expired user it will prompt a password change.
  
  
• Disabling LDAP temporarily can immediately stop the ongoing issue.