Scheduler Messaging Regarding Agent Disk Space Monitoring
Article ID: 243822
Updated On:
Products
Issue/Introduction
When the agent is configured to monitor the amount of disk space available in the agent's 'database' directory, it will send alerts to the AutoSys Scheduler when the various thresholds are breached. This document explores the various messages that will appear in the Scheduler log pertaining to this functionality.
Environment
Release : 11.3.6 SP8/12.0/12.0 SP1
Component :AutoSys Workload Automation
Resolution
When the agent disk space monitoring is enabled, there are three configured thresholds...
NOTICE
SEVERE
CRITICAL
When the available space on the agent falls below the NOTICE threshold, the following messages will appear in the Scheduler log...
[06/14/2022 12:16:54] CAUAJM_I_40245 EVENT: ALERT MACHINE: agent_machine
[06/14/2022 12:16:54] <Type(DISK) Status(WARNING) SEVERITY(WARNING) CRITICALEVENT(false) Description(Disk resource below threshold.) DiskSpaceCurrent(58G) DiskSpaceThreshold(60G)>
[06/14/2022 12:16:54] CAUAJM_I_40245 EVENT: ALARM ALARM: MACHINE_DISKTHRESHOLD MACHINE: agent_machine
[06/14/2022 12:16:54] <Disk resource below threshold. Machine <agent_machine> communication is active. Status(WARNING) DiskSpaceCurrent(58G) DiskSpaceThreshold(60G)>
The messages that appear in the Scheduler log for the SEVERE and CRITICAL thresholds will depend on the value set for this agent configuration parameter...
agent.resourcemon.threshold.disk.critical.shutdown
Options for this parameter...
false (default if not set) - The agent will continue to run persisting to memory when the critical disk threshold has been breached.
true - The agent will start blocking all incoming job requests when the SEVERE threshold is breached. The agent will shutdown completely when the CRITICAL threshold is breached.
When this configuration parameter is set to 'false', the messages that appear in the Scheduler log for the SEVERE and CRITICAL threshold breaches are as follows...
SEVERE...
[06/14/2022 12:18:22] CAUAJM_I_40245 EVENT: ALERT MACHINE: agent_machine
[06/14/2022 12:18:22] <Type(DISK) Status(SEVEREWARNING) SEVERITY(SEVEREWARNING) CRITICALEVENT(true) Description(Disk resource below severe threshold.) DiskSpaceCurrent(57G) DiskSpaceThreshold(58G)>
CRITICAL...
[06/14/2022 12:19:53] CAUAJM_I_40245 EVENT: ALERT MACHINE: agent_machine
[06/14/2022 12:19:53] <Type(DISK) Status(SEVEREWARNING) SEVERITY(CRITICAL) CRITICALEVENT(true) Description(Disk resource below critical threshold. Agent is entering persist to memory mode.) DiskSpaceCurrent(54G) DiskSpaceThreshold(55G)>
When the shutdown parameter is set to 'true', the messaging in the Scheduler log for SEVERE and CRITICAL threshold breaches are as follows...
SEVERE
[06/14/2022 12:27:32] CAUAJM_I_40245 EVENT: ALERT MACHINE: agent_machine
[06/14/2022 12:27:32] <Type(DISK) Status(BLOCKED) SEVERITY(SEVEREWARNING) CRITICALEVENT(true) Description(Disk resource below threshold. Blocking receiver) DiskSpaceCurrent(60G) DiskSpaceThreshold(61G)>
[06/14/2022 12:27:32] CAUAJM_W_40258 Machine <agent_machine> has explicitly rejected communication. Placing in a blocked state.
[06/14/2022 12:27:33] CAUAJM_I_40245 EVENT: ALARM ALARM: MACHINE_DISKTHRESHOLD MACHINE: agent_machine
[06/14/2022 12:27:33] <Disk resource below threshold. Machine <agent_machine> has blocked communication. Status(BLOCKED) DiskSpaceCurrent(60G) DiskSpaceThreshold(61G)>
[06/14/2022 12:27:33] CAUAJM_I_40245 EVENT: ALARM ALARM: MACHINE_UNAVAILABLE MACHINE: agent_machine
[06/14/2022 12:27:33] <Machine <agent_machine> placed in a blocked state due to explicit message rejection or invalid agent configuration.>
CRITICAL
[06/14/2022 13:11:23] CAUAJM_W_40293 Machine <lvntest020808> is not responding or has shut down. Automatically placing offline.
[06/14/2022 13:11:23] CAUAJM_I_40245 EVENT: ALERT MACHINE: agent_machine
[06/14/2022 13:11:23] <Type(DISK) Status(SHUTDOWN) SEVERITY(CRITICAL) CRITICALEVENT(true) Description(Disk resource below critical threshold. Agent is shutting down.) DiskSpaceCurrent(59G) DiskSpaceThreshold(60G)>
[06/14/2022 13:11:23] CAUAJM_I_40245 EVENT: ALARM ALARM: MACHINE_DISKTHRESHOLD MACHINE: agent_machine
[06/14/2022 13:11:23] <Disk resource below threshold. Machine <agent_machine> has shutdown. Status(SHUTDOWN) DiskSpaceCurrent(59G) DiskSpaceThreshold(60G)>
[06/14/2022 13:11:24] CAUAJM_I_40245 EVENT: ALARM ALARM: MACHINE_UNAVAILABLE MACHINE: agent_machine
[06/14/2022 13:11:24] <Machine <agent_machine> is not responding or has shut down.>
If the space issue is resolved while the agent is either persisting in memory or is in a blocked state but has not actually shut down, the following messages will appear in the Scheduler log...
[06/14/2022 12:28:33] CAUAJM_I_40245 EVENT: MACH_ONLINE MACHINE: agent_machine
[06/14/2022 12:28:33] <Disk threshold breach resolved. Machine agent_machine communication is active. Placing machine online.>
[06/14/2022 12:28:33] CAUAJM_I_40120 Completed 0 job start(s) for online machine <agent_machine>.
[06/14/2022 12:28:34] CAUAJM_I_40245 EVENT: ALERT MACHINE: agent_machine
[06/14/2022 12:28:34] <Type(DISK) Status(NORMAL) SEVERITY(INFO) CRITICALEVENT(false) Description(Disk resource no longer breaching any thresholds) DiskSpaceCurrent(62G) DiskSpaceThreshold(0B)>
If the agent was shutdown prior to resolving the space issue, only a MACH_ONLINE alarm will appear when the agent is restarted.
NOTE: Any message containing "EVEVT: ALARM" in the above scenarios will also be visible in the WebUI(WCC) if they are captured by an alert policy. The messages containing "EVENT: ALERT" will only appear in the Scheduler log.
Additional Information
For details about configuring an agent for disk space monitoring, see the following link...
https://techdocs.broadcom.com/us/en/ca-enterprise-software/intelligent-automation/workload-automation-system-agent/12-0/configuring/configure-the-agent/configure-the-agent-to-monitor-available-disk-space.html
Feedback
