Search Server indexes in a cluster are not being generated

Products

• CA Service Management - Service Desk Manager
• CA Service Desk Manager

Issue/Introduction

When using Service Point, finding that Catalog offerings are not appearing and Service Point tickets are not appearing either.

When using the ElasticSearch head to examine the backend Search Server, finding that none of the indexes are updating and that some indexes are missing.  Running pdm_es_rebuild_index.bat has no effect.

Issue is appearing when there are two Search Servers installed for the same environment, operating in a cluster

Environment

Release : 17.3

CA Service Management:  xFlow

Cause

Configuration in both Search Servers' elasticsearch.yml needs to be re-examined.

Resolution

The elasticsearch.yml file will be in folder C:\Program Files\CA\SC\CASearchServer\elasticsearch-XXXX\config where XXXX is the latest version of Search Server that was installed.

If you want the two search servers to run independently, then the configuration in the elasticsearch.yml file on each server should look like this:
node.name: <<HOST_NAME>>
network.host:<<HOST_NAME>>
cluster.name: ca_es_cluster
cluster.initial_master_nodes: <<HOST_NAME>>
discovery.seed_hosts:<<HOST_NAME>>
http.port: 9012

For example, if the hostnames of the two servers are SearchServer1 & SearchServer2

On the SearchServer1 server it should be:

node.name: SearchServer1
network.host:SearchServer1
cluster.name: ca_es_cluster
cluster.initial_master_nodes: SearchServer1
discovery.seed_hosts:SearchServer1
http.port: 9012

On the SearchServer2 server it should be:

node.name: SearchServer2
network.host:SearchServer2
cluster.name: ca_es_cluster
cluster.initial_master_nodes: SearchServer2
discovery.seed_hosts:SearchServer2
http.port: 9012

If you want to configure the two search servers to work as an elastic cluster, then

node.name:<<HOST_NAME>>
node.master: true
node.data: true
network.host:<<HOST_NAME>>
cluster.name: ca_es_cluster
cluster.initial_master_nodes:<<MASTER_HOST_NAME>>
discovery.seed_hosts: [ MASTER_ELIGIBLE_HOST_NAMES]
http.port: 9012

"node.master" means eligible master node, we should set this value to true for all the nodes in the cluster.
"discovery.seed_hosts", this should have list of all the eligible master nodes(means all the nodes in the cluster)

But cluster.initial_master_nodes should have only master node

if you want to make SearchServer1 the master node then the configuration should look like this:

On the SearchServer1 server it should be:

node.name:SearchServer1
node.master: true
node.data: true
network.host:SearchServer1
cluster.name: ca_es_cluster
cluster.initial_master_nodes:SearchServer1
discovery.seed_hosts: [ SearchServer1,SearchServer2]
http.port: 9012

On the SearchServer2 server it should be:

node.name:SearchServer2
node.master: true
node.data: true
network.host:SearchServer2
cluster.name: ca_es_cluster
cluster.initial_master_nodes:SearchServer1
discovery.seed_hosts: [ SearchServer1,SearchServer2]
http.port: 9012

After doing the above changes, you should delete all the indices from both the servers.

Once the indexes have been deleted from both servers, if you are configuring an elastic cluster, you will need to run rebuild index only the server that is the master node.  The elastic cluster will take care of updating the data and syncing the data in both the nodes.

If you are not configuring an elastic cluster, you need to run the rebuild index on both the servers, one after the other.