Connectivity check failed: Could not find accessible execution server for agent

Products

CA Release Automation - Release Operations Center (Nolio)

Issue/Introduction

While running deployments the following error is frequently seen:

Name: <Step Name>

Status: Initialization Failed 0%

At the bottom of the screen it shows “Deployment Errors”

Step: <Step Name>

Error: <Step Name> has a process error - Connectivity check failed. reason: Could not find accessible execution server for agent [<Agent Hostname>:<ip_address>], Agent may be offline.

The error described above reflects the error as it was shown within the Deployment phase. However, though it could happen in the Pre-Deployment or Post Deployment phases too. It can also happen while trying to handle Artifact Distribution phase of a release and even during the Pre-Plan step of building deployment plans.

NOTE:

This is different from the errors:

Error: <Process Name> has a process error - Connectivity check failed
Error: <Process Name> has a process error - Deployer on host <Agent hostname> does not respond.
Error: <Process Name> has a process error - Server <Agent hostname>(Agent IPAddress) error

If you're seeing these errors then please review this KB Article: Connectivity check failed: Deployer on host <agentname> does not respond

Cause

The underlying cause is that the Management Server believes that the Execution Server is unreachable. Some causes for this can include:

The Execution Server is not started.
The Management Server cannot communicate with the Execution Server via the ports that the Execution Server listens on.
The Execution Server and/or Management Server are having activemq problems.
SQL Performance problems.

This article will cover some things you can do for items 1-3 in the Additional Information section. The Resolution section of this article covers some configuration changes that can help prevent these errors when they are caused by #4 - SQL Performance problems. If you have many Execution Servers then this problem can also be observed with many, or all, execution servers becoming unavailable.

Environment

Release : 6.6

Component : CA RELEASE AUTOMATION CORE

Resolution

If the SQL Server is experiencing performance problems then the availability of Execution Servers is one, of several, problems that might be observed. If the SQL Server performance problems are the cause of these Execution Servers becoming unreachable then increasing the following configuration settings may help.

Notes/Recommendation:

It is a best practice to set the fixed-delay value to half the value of the nes.expiration value.
These changes need the management server and execution server's services to be stopped and started.

There are two configuration settings. One configuration setting is made on the management servers. Once configuration setting is made on the execution server.

Management Server

File: NOLIO_HOME/webapps/datamanagement/WEB-INF/numeric.properties:

Configuration Entry: nes.expiration=<value in ms>

Default Value: 60000 (1 minute)

Execution Server

File: NOLIO_HOME/webapps/execution/WEB-INF/outbound-nes.xml

Configuration Entry: fixed-delay="<value in ms>"

Default Value: 30000 (30 seconds)

Additional Information

Additional Information For Causes 1-3:

The Execution Server is not started

If the Execution Server is not started then start the execution server.

The Management Server cannot communicate with the Execution Server via the ports that the Execution Server listens on

If the management server cannot communicate with the Execution Server using the ports that the Execution Server listens on then you will need to speak with your Network and/or SysAdmin Teams to address any firewall issues that might be causing this connectivity problem.

The Execution Server and/or Management Server are having activemq problems

Some of these problems are further described/addressed via these KB Articles:

Execution Servers Are Not Connecting

Execution Server Not Working

CA Release Automation Clean up and Troubleshooting best practices