Error: OPS3146S EMERGENCY PRODUCT SHUTDOWN STARTED - MAXIMUM WTO RATE EXCEEDED

Products

• OPS/MVS Event Management & Automation

Issue/Introduction

The OPS/MVS Event Management and Automation task terminates with the message OPS3146S EMERGENCY PRODUCT SHUTDOWN STARTED - MAXIMUM WTO RATE EXCEEDED. This message indicates that OPS/MVS automatically triggered an emergency shutdown because Write-To-Operator (WTO) rate limits were reached. This safety mechanism protects the z/OS system from resource depletion when a sudden burst of messages or an automation loop generates high message volume.

 

Environment

• Product: OPS/MVS Event Management & Automation for JES2
• Relevant Parameters: MESSAGEMAX, MESSAGERATE

Cause

The MESSAGEMAX threshold is reached due to excessive WTO activity. Common causes include:

• A runaway automation rule or REXX process (e.g., an OPSUSS server event) firing in a loop.
• A sudden burst of system messages reacting to an external event.
• Incorrect threshold configuration relative to the environment's message volume.

Resolution

1. Identify the Cause:

   • Analyze the OPSLOG archive (1 hour before/after the event) to identify the specific jobs, REXX rules, or messages firing at high frequency.
   • Look for recursive automation calls or repeating messages that trigger the shutdown.

2. Monitor Thresholds:

   • Use the OPSVIEW panel (Option 4.1.1) to monitor current usage of MESSAGEHIGH
   • Compare these metrics against your defined MESSAGEMAX and MESSAGERATE parameters.

3. Tune Parameters:

   • If the volume is legitimate for your environment, increase the MESSAGEMAX threshold.
   • Directly in the OPSVIEW panel 4.1.1
     
   • Using theTSO Command: TSO OP SET(MESSAGEMAX) VALUE(10000)
   • Updating the OPSSPAxx member: T = OPSPRM('SET', 'MESSAGEMAX', '10000')
   • Note: Always validate threshold changes with your system engineering team to prevent unintended resource exhaustion.

4. Automation Correction:

   • If a runaway loop is identified, disable or modify the specific automation rule to prevent infinite recursion.