A dSeries job executes multiple times under the same application generation.

Products

• ESP dSeries Workload Automation
• Workload Automation Agent

Issue/Introduction

In high-availability environments, users often attempt to utilize external Load Balancers or Virtual IP (VIP) addresses to distribute workload across multiple CA Workload Automation (CAWA) agents. However, this architecture can lead to critical synchronization failures. When an external network appliance manages the traffic, the direct communication channel between the scheduler and the executing agent is masked. If a network flicker, node health-check failure, or high CPU utilization causes the Load Balancer to shift the session to a different backend node, the scheduler may interpret the loss of the original connection as a submission failure. Consequently, the scheduler attempts to recover the job by resubmitting it through the VIP, which can result in the same job generation executing concurrently or sequentially on two different physical nodes.

Symptoms:

• Duplicate Execution: The transmitter.archive.log (or equivalent agent logs) reflects two distinct State EXEC SetStart and State COMPLETE SetEnd cycles for a single job instance/generation.
• Log Discrepancy: Different Jobno, ChildPid, or Host identifiers are recorded for the same application generation within a short timeframe.
• Ghost Submissions: No manual resubmission or re-run commands are found in the scheduler's history or audit logs, indicating an automated or architectural trigger for the second run.

Environment

Product: ESP dSeries Workload Automation, Workload Automation Agent
Version: All versions
Architecture: External VIP/Load Balancer (e.g., F5, Citrix ADC, AWS NLB) positioned between the Scheduler/Manager and multiple active Agent nodes.

Cause

Broadcom Workload Automation solutions are designed for a 1-to-1 persistent connection between the scheduler and the agent. Using an external Load Balancer breaks this logic because:

1. State Tracking: The scheduler tracks the state of a job based on the specific connection to the agent node. A VIP masks which node is actually active, preventing the scheduler from accurately confirming if the first execution completed before the second was initiated.
2. Inappropriate Failover: External load balancers operate at the network layer and are unaware of the application-level state of a running job. They may redirect a retry attempt to a new node while the first node is still processing the original task.

Resolution

To prevent duplicate executions and ensure supported tracking, decommission the external VIP for agent traffic and utilize native Scheduler load-balancing features.

Recommended Best Practices

1. Define Unique Real Machines: Ensure each physical or virtual agent node is defined as a unique Real Machine in the scheduler using its dedicated hostname or IP.
2. Configure Agent Groups (Virtual Machines): Create a Virtual Machine (Agent Group) definition within the scheduler and add the Real Machines as members.
3. Implement Native Load Balancing: Set the Selection criteria to CPU, Random or Round Robin within the Virtual Machine definition. This allows the scheduler to maintain the required 1-to-1 connection for the duration of the job's lifecycle.
4. Update Job Targets: Reconfigure application/job definitions to target the Virtual Machine name instead of the VIP address.

Additional Information

To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on the respective region.