Service Point Connection Refused requiring Service Desk Manager Services recycle

Products

• CA Service Management - Service Desk Manager
• CA Service Catalog

Issue/Introduction

Intermittent connection issue between Service Desk Manager, Service Point Apps and Service Catalog. SDM Services was being recycled by executing ("net start pdm_daemon_manager") command to start it every two days or sometimes daily

incidentMS Log file shows the following messages and also error messages:

WARN - 2021-06-30 09:59:47 [a.remote.ReliableDeliverySupervisor] - Association with remote system [akka.tcp://Incident_MicroService_XflowClusterSystem@SDM14:2551] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://Incident_MicroService_XflowClusterSystem@SDM14:2551]] Caused by: [java.net.ConnectException: Connection refused: no further information: SDM/IP address:2551]
WARN - 2021-06-30 09:59:51 [akka.cluster.JoinSeedNodeProcess] - Couldn't join seed nodes after [24779] attempts, will try again. seed-nodes=[akka.tcp://Incident_MicroService_XflowClusterSystem@SDM:2551]
WARN - 2021-06-30 09:59:56 [akka.cluster.JoinSeedNodeProcess] - Couldn't join seed nodes after [24780] attempts, will try again. seed-nodes=[akka.tcp://Incident_MicroService_XflowClusterSystem@SDM:2551]
WARN - 2021-06-30 09:59:57 [a.r.transport.netty.NettyTransport] - Remote connection to [null] failed with java.net.ConnectException: Connection refused: no further information: SDM/IP address:2551
WARN - 2021-06-30 09:59:57 [a.remote.ReliableDeliverySupervisor] - Association with remote system [akka.tcp://Incident_MicroService_XflowClusterSystem@SDM14:2551] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://Incident_MicroService_XflowClusterSystem@SDM14:2551]] Caused by: [java.net.ConnectException: Connection refused: no further information: SDM/IP address:2551]
WARN - 2021-06-30 10:00:01 [akka.cluster.JoinSeedNodeProcess] - Couldn't join seed nodes after [24781] attempts, will try again. seed-nodes=[akka.tcp://Incident_MicroService_XflowClusterSystem@SDM:2551]
WARN - 2021-06-30 10:00:06 [akka.cluster.JoinSeedNodeProcess] - Couldn't join seed nodes after [24782] attempts, will try again. seed-nodes=[akka.tcp://Incident_MicroService_XflowClusterSystem@SDM:2551]

Environment

Release : 17.2 +

Component : SERVICE DESK MANAGER

Cause

 Service Desk Manager  Server's TCP/IP stack was configured to obtain an IP Address automatically which allows any networking device (DHCP) to assign a dynamic IP Address to it.

There is also a host files configured on the servers. Process sequence is such that  the host file on the Operating Systems overrides whatever configured on the DNS server. Make sure that this is a desired behavior and the FQDN is configured along with the IP address in the host file.

Resolution

While this may not be a problem and can be managed via DHCP reservation, each time the lease period expires and the IP renewal requested, the services provided by the server will experience brief interruptions. This is resulting in "Connection refused" messages.

Recommended that change SDM and Service Catalog servers TCP/IP configuration from auto to Static IP addresses and this has resolved the  intermitent Service Point issue

Customer  made aware that  the host file on the Operating Systems always override the configuration settings on the DNS server. If the host file is the standard configuration, the FQDN name must be configured along with the IP address in the host file.