Dollar Universe fails to start via Pacemaker in a Linux Azure Cluster: Cannot assign requested address

Products

• CA Automic Dollar Universe

Issue/Introduction

On two Azure Cloud Linux Redhat Servers with Pacemaker as Cluster tool, the virtual hostname remapping cannot be done at startup when failing over from node A to node B.

Hence it is impossible to start Dollar Universe in cluster mode with a virtual hostname

Error : 

| 2022-10-10 09:52:55 |ERROR|X|IO |pid=p.t| u_ouv_serv                | failed in u_listen: Errno syserror 99: Cannot assign requested address
| 2022-10-10 09:52:55 |FATAL|X|IO |pid=p.t| u_io_srv_main             | Error opening IO server X service
| 2022-10-10 09:52:55 |ERROR|X|IO |pid=p.t| u_io_srv_main             | COM Error u_listen -1: Errno syserror 99: Cannot assign requested address,bind(socket 8) returns error

Environment

Release : 6.x and 7.x

OS: RHEL 8 Cluster with two servers on Azure Cloud

Cluster tool: Pacemaker

Cause

The virtual hostname is not added to the servers dynamically on Azure Linux Servers when performing a Failover as on other Cluster architectures such as Windows Cluster or Veritas Cluster.

Resolution

The issue was fixed by changing the variable U_LOCALHOSTNAME before starting the node on the Failover server, this is done with the commmand:

$UNI_DIR_EXEC/unims -update -host new_hostname_failover_server

This is done on the Pacemaker module in Linux to manage the failover, this should be done before attempting to start the node on the failover server.

Example of the configuration done in Pacemaker to integrate Dollar Universe in an Azure Cloud:

Three resources were defined in Pacemaker
 Resource Group: BATCH_RES
     dollardata_fs      (ocf::heartbeat:Filesystem):    Started
     dollarulog_fs      (ocf::heartbeat:Filesystem):    Started 
     dollaru_ser        (service:cip-DollarU-NSPTRN.service):  Started 

dollardata_fs and dollarulog_fs: unmount/mount Filesystems for Dollar Universe data and log 
dollaru_ser : resource to update values.xml and node declaration with the newer hostname and to stop/start Dollar Universe

Pacemaker script: 

#!/bin/bash
if [ $1 == 'start' ]
then
  source /dollaruniverse_folder/unienv.ksh
  U_HOSTNAME=`hostname -s`
  $UNI_DIR_EXEC/unims -update -host $U_HOSTNAME
  $UNI_DIR_EXEC/unistart
  elif [ $1 == 'stop' ]
then
  source /dollaruniverse_folder/unienv.ksh
  $UNI_DIR_EXEC/uxstpque queue=* next
  $UNI_DIR_EXEC/unistop
fi