Crowdstrike Falcon July 2024 Incident - impact on DX UIM

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

A recent incident with Crowdstrike July 19 2024 has led to widespread server outages/crashes/reboots.

This may impact DX UIM in one of the following ways:

- Operator Console/wasp may fail to start
- Unable to log into primary hub or start it
- primary hub does not start all probes
- some probes do not start correctly and turn red

 

 

Environment

Any release

Windows OS

 

Cause

Crowdstrike incident (July 19 2024)

Resolution

DX UIM is not directly impacted by Crowdstrike.

However, at times, the crash/reboot cycle caused by this incident may impact DX UIM environments.

The first step after recovery would be to stop the UIM Services on the primary hub and operator console robot(s), and then restart the primary hub robot first, then Operator Console robot(s) after it fully starts.

This may resolve any issues that were caused by loss of connectivity to the database or between the UIM Servers.

If the primary hub does not start properly or only starts the controller probe this can be due to corruption of the controller.cfg.

If this is the case you will be unable to log into DX UIM or Operator Console.

The following KB can be used to help recover from this:

Cannot connect to my hub and my controller.cfg is missing probes

In other cases it has been observed that specific probe .cfg files have been impacted/corrupted (example NAS, EMS, alarm_enrichment) which may cause these probes not to start.

It may be sufficient to restore the .cfg structure by simply re-deploying the probe(s) which have been impacted (take a backup copy of the probe .cfg first just in case) --  but in some cases it may be necessary to restore the configuration files from a backup.

Additional Information

https://www.crowdstrike.com/blog/statement-on-falcon-content-update-for-windows-hosts/