Rebuild the Elastic Search Index for Cloud Data Protection

book

Article ID: 168876

calendar_today

Updated On:

Products

CDP for ServiceNow CDP for Salesforce

Issue/Introduction

The goal of this article is to give complete instructions on how to rebuild your local search index in  4.x environment if you are a experiencing unresolvable issues with local search.

Environment

Cloud Data Protection (CDP) 4.x

Resolution

Steps:

  1. Remove data sync job from job management
  2. Delete local search index
  3. Restart the Search module
  4. Perform a refresh metadata

1. Remove data sync job from job management

  1. In the Management Console click "Diagnostics" -> "Jobs Management" in the side panel.
  2. For each instance there should be a refresh metadata, and data sync job.
  3. For the search index you wish to rebuild delete the data sync job by clicking the red X button in the actions column.

2. Delete local search index

Through Management Console

Pre-requisite: Obtain the environment ID and instance ID related to the search index you wish to delete.

  1. In the Management Console navigate to "Diagnostics" -> "API Documentation"
  2. Expand the row for Search.
  3. There should be DELETE method, expand this and enter in the required information.
  4. Once you entered environment and instance ID proceed by clicking "Try it out".
  5. Below this it should indicate if it succeeded. 

Alternative via manual deletion

  1. Navigate to the Elastic Search data directory.  You can find this in the Search module configuration under "Elastic Search Configuration" --> "Data directory", but typically it is a folder named "Data" in the main container folder (/usr/local/perspecsys/container).
  2. Within that folder should be a folder for the cluster.  This will be named after the cluster, you can find the name, again, from "Elastic Search Configuration".
  3. Once in there should be a sub folder named "nodes".  Typically there will only be one node subfolder named "0" enter this folder.  Once in there there will be a folder named "indices" that contains all the search indices, enter this.
  4. For each search index there will be a separate folder.  Delete the folder relating to the search index you are having difficulties with.

Alternative delete via Elastic Search REST API

If you are unsure where the folder for Elastic Search is or you don't have access you can also delete the index via REST provided  -Dsearch.rest.enabled=true for the container jar.  Check the start.sh script
  1. First retrieve all search indices by performing

    curl '<container address>:<search port>/_cat/indices?v'

    e.g

    curl 'localhost:9200/_cat/indices?v'
  2. This should list the search indices available.  Find the one you wish to delete
  3. Delete by performing

    curl -XDELETE 'http://<container address>:<search port>/<index>/'


    e.g.

    Elastic search should reply with a JSON confirming deletion

3. Restart the Search module

For Elastic Search to rebuild the search index correctly I've found a restart of the Search module is necessary/sufficient.  Note: in some environments it has been observed that simply restarting the Search module is not enough and you must restart the container, if there are issues during refresh metadata (e.g. 504 errors) this may be the case.

To restart module

  1. Navigate to the Management Console Dashboard. 
  2. For Search module click the status drop down menu (probably shows "Started")
  3. click restart module
  4. wait until you see a notification that Search module is back up

To restart the container or cluster manager (For CDP 4.12.6 or older)

Either restart the services or use the restart script in the bin folder of the container.

4. Perform a refresh metadata

  1. In the Management Console click "Diagnostics" -> "Jobs Management" in the side panel.
  2. For the instance search index you wish to rebuild you'll need to execute the corresponding refresh metadata job (play button), once complete this will create a new data sync job.

Troubleshooting

If you are still getting local search index issues after rebuilding the local search index you may have wider issues with Elastic Search.  One thing to try would be stopping all data sync jobs for all instances, deleting entire Elastic Search data folder, restarting search, performing refresh metadata for all instances.