UIM cdm disk full alarms not generated after crossing set threshold value
Article ID: 259408
Updated On:
Products
Issue/Introduction
Why didn't CDM alarm when our disk filled up?
Environment
Release: 20.1, Any release, Any OS
Component: cdm probe: any release
Cause
When a disk fills up rapidly due to a core or crash file and fills the hard drive, cdm, unless the interval query was running, and there was still some disk space, by the time it runs, the disk will be full and the cdm probe will be unable to write the alarm to the log and the queue. Not being able to write to the file system will prevent the alarm from being sent.
This can happen with any monitoring software/solution and any OS.
Resolution
One preventive measure that could be taken is to install and keep the robot on a separate drive/file system to the application that potentially will fill the storage, but that is not always possible as there may only be a single drive or filesystem available.
Solutions may include one or more of the following:
- Proactive monitoring which includes baselining and thresholding including ever-increasing alarms based on stepped percentage-climb
- Dedicated response to an earlier alarm by the systems administrator/systems team / manual intervention
- Automatic remediation via nas ao profile using scripts/commands to temporarily free up some space on the given drive/filesystem and send a critical alarm via email and text
If the application on the same system is considered 'critical,' then that application should be on its own file system and separate from the UIM robot.
If the application on the same system is less critical, despite this fact, the robot won't be able to report to its hub and a robot inactive alarm will be thrown by the hub.
Feedback
