K8S_CMD Jobs Failing Due to Premature Pod Termination in Kubernetes

Products

• Autosys Workload Automation
• ESP dSeries Workload Automation
• ESP dSeries Workload Automation - Business Agents (dSeries)
• ESP dSeries Workload Automation - Scheduler (dSeries)
• ESP dSeries Workload Automation - System Agent (dSeries)
• ESP Workload Automation
• Workload Automation Agent

Issue/Introduction

Intermittent status mismatches are observed between Autosys and Kubernetes jobs in environments where the Kubernetes environment uses default configuration settings. Autosys jobs may consistently complete in Kubernetes, but Autosys does not reflect the correct status. Additionally, Autosys agent pods may show frequent restarts or short uptime, which prevents reliable job completion signaling.

Environment

• Autosys Workload Automation
• Workload Automation Agent with Kubernetes Plugin Extension
• Autosys Agents deployed in Kubernetes (K8S_CMD jobs)

Cause

The terminationGracePeriodSeconds setting in the Kubernetes environment is set to the default value of 30 seconds. This period is insufficient for long-running workloads or tasks that require additional time to report completion back to Autosys. Consequently, the agent pods are terminated by the Kubernetes scheduler before the job status can be successfully transmitted.

Resolution

To ensure reliable job status synchronization, update the terminationGracePeriodSeconds configuration in the affected Kubernetes environments.

1. Access your Kubernetes deployment configuration.
2. Increase the terminationGracePeriodSeconds value to a duration sufficient for your longest expected workload (e.g., 2 hours).
3. Apply the configuration change and restart the agent pods.
4. Monitor the agent pod uptime and verify that Autosys reflects the correct "Completed" status for jobs once they finish.

Additional Information

To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on the respective region.