AutoSys notification attribute is not sending the whole log file as email attachment

Products

• Autosys Workload Automation

Issue/Introduction

There is an issue with a job using an email template to send a large log file for the job as an email attachment to the notification. However, it is only sending part of the file.

The glob has LOG_LINES set to 0 and the LogMaxEndLines is set to 0 in each application server.

The  app server has been restarted to enforce the settings. Only the first 2000 lines are being sent, with the remaining being truncated.

Cause

Due to product limitation to protect the scheduler from crashing.

In AutoSys 12.1, a change was introduced to limit the maximum number of lines in an email attachment to prevent scheduler crashes. The maximum allowed is 10000 lines.

Examples:

    // The value of LOG_LINES and LogMaxEndLines will determine
    // the value of number of lines to be retrieved
    // LOG_LINES             | LogMaxEndLines         | Return Value|
    // ==============================================================
    // 0 (full file)         | 0 (full file)          |    2000     |
    // 0 (full file)         | 10000 (MAX Allowed)    |   10000     |
    // 0 (full file)         | 10001 (Over the limit) |    2000     |
    // 0 (full file)         | 500                    |     500     |
    // 500                   | 400                    |     400     |
    // 400                   | 500                    |     400     |
    // No Val                | 0 (full file)          |    2000     |
    // 500                   | 0 (full file)          |     500     |
    // 10000 (MAX Allowed)   | 0 (full file)          |   10000     |
    // 10001(Over the limit) | 0 (full file)          |    2000     |
    // 10000 (MAX Allowed)   | 10000 (MAX Allowed)    |   10000     |
    // 10001(Over the limit) | 10001 (Over the limit) |    2000     |
    // ==============================================================

Resolution

Viable Workaround Solutions:

• Workaround Category 1: Bypass the AutoSys Notification Engine (Recommended)
  The most robust solution is to stop using the built-in AutoSys email template for the attachment. Instead, add a step within the job workflow itself to handle the email sending using OS-level tools. AutoSys simply triggers the command that sends the email.
  
  
  • Solution 1.A: OS-Level Command Line Emailers (mailx, sendmail, Blat)
    If the AutoSys agent is running on Linux/Unix, tools like mailx or sendmail are likely already available. If on Windows, tools like Blat or PowerShell can be used.
    
    
    • How it works:
      1. Job A runs and generates the 9MB output file (e.g., /tmp/large_report.txt).
      2. Add a subsequent step (or a dependent Job B) that executes a command line script.
      3. This command sends the email directly via the company's SMTP server, attaching the file without involving the AutoSys notification engine constraints.
         
         Example Command (Linux):
         Bash
# This runs as a command job in AutoSys
echo "Here is the requested report." | mailx -s "Daily 9MB Report" -a /tmp/large_report.txt [email protected]
         
         
         • Pros: Highly reliable; bypasses all AutoSys attachment limits; standard operational practice.
         • Cons: Requires SMTP relay access from the agent machine; requires slight modification to the job definition (adding a step).
           
           
  • Solution 1.B: Scripting Languages (Python, PowerShell, Perl)
    For more complex email requirements (HTML bodies, multiple attachments, specific SMTP authentication), a small script is better than a raw command line.
    
    
    • How it works: Similar to 1.A, AutoSys triggers a script (e.g., send_report.py or send-report.ps1) after the main job finishes. This script uses standard libraries to construct an MIME email with the 9MB attachment and send it.
      
      
      • Pros: Extremely flexible; platform-independent (if using Python/Perl).
      • Cons: Requires maintaining a small external script.
        
        
• Workaround Category 2: The "Pointer" Method (Modern Best Practice)
  Sending 9MB attachments via email is generally discouraged in modern IT environments due to inbox bloat, network strain, and security filters. The best practice is to store the file securely and email a link to it.
  
  
  • Solution 2.A: Save to Network Share (UNC Path)
    
    
    • How it works:
      1. The AutoSys job generates the file and saves it to a secured, shared network location (e.g., \\fileserver\reports\daily\).
      2. The standard AutoSys email notification template is used.
      3. Instead of attaching the file, the email body contains a hardcoded link or variable pointing to the location.
         
         Template Example: "Job XYZ has completed. The 9MB output file is too large for email attachment. Please access the report here: \\fileserver\reports\daily\output_20231027.txt"
         
         
         • Pros: No attachment limits; keeps email system clean; uses existing file server security permissions.
         • Cons: Users need network access to the share location (might be an issue for remote/mobile users).

• • Solution 2.B: Upload to Cloud/Web Storage (S3, Azure Blob, SharePoint)
    If the organization uses cloud infrastructure, this is the most scalable approach.
    
    
    • How it works:
      1. The job generates the file.
      2. A subsequent job step uses CLI tools (like aws s3 cp or azcopy) to upload the file to a cloud storage bucket.
      3. The job generates a pre-signed URL (a temporary, secure link) to the file.
      4. AutoSys sends a standard notification email containing that URL.
         
         
         • Pros: Highly scalable; accessible from anywhere (if permitted); secure.
         • Cons: Requires cloud infrastructure setup and appropriate CLI tools installed on the agent.
           
           
•  Workaround Category 3: Mitigation (Least Likely to Succeed)
  
  • Solution 3.A: Compression before Attaching
    If the 9MB file is highly compressible text (like a log file), zipping it might bring it down to 1-2MB.
    
    • How it works: Add a job step to run gzip filename.txt before the notification triggers. Configure AutoSys to attach the .gz file.
    • The Catch: While this solves the size issue for the mail server, it might not solve the AutoSys internal limitation. If AutoSys tries to open the attachment to check line counts (even if binary), it may still reject it. Furthermore, many corporate email filters block zipped attachments for security reasons.
      
      
      • Pros: Minimal change to workflow.
      • Cons: High chance of failure depending on how AutoSys enforces its limits; attachments may be blocked by spam filters.

Additional Information

notification_template -- Specify the Template for the Email Notification