Search Server Troubleshooting
Article ID: 225742
Updated On:
Products
Issue/Introduction
Search Server, or Elastic Search is a component that is leveraged by xFlow and Service Point to access content from Service Desk and Service Catalog. The following is a guide to how the Search Server component operates and what to look for in troubleshooting.
This article will be of value when troubleshooting scenarios such as missing Service Desk ticket content in Service Point as well as missing Catalog offering data in Service Point.
Environment
Release : 17.3
Component : SDM - xFlow
Resolution
Table of Contents
Search Server logging;
Logging for Search Server/ElasticSearch is contained in the ca_es_cluster.log log file, which is contained in the Search Server install directory's logs directory. Depending on the release, the default directory locations are:
- Prior to SM 17.3 RU6, the file is located in C:\Program Files\CA\SC\CASearchServer\elasticsearch-2.1.1\logs
To set the logging into debug mode, modify the line "es.logger.level: INFO" to "es.logger.level: DEBUG" in elastic search server install\elasticsearch-2.1.1\config\logging.yml file and recycle Search Service - From SM 17.3 RU6 onward, the file is located in C:\Program Files\CA\SC\CASearchServer\elasticsearch-7.10.2\logs
To activate debugging, locate file "log4j2.properties" in C:\Program Files\CA\SC\CASearchServer\elasticsearch-7.10.2\config. Backup, then edit the file in a text editor and locate this line:
rootLogger.level = info
Change the line to read as:
rootLogger.level = DEBUG
Save the "log4j2.properties" file, then recycle Search Service
When performing an initial load (pdm_es_initial_load.bat) or a subsequent index rebuild (pdm_es_rebuild_index.bat), file initial_load.log will be of value. This log file can be located in the C:\Program Files\CA\SC\CASearchServer\search\logs directory.
Service Desk/Search Server Interaction
Within SDM, under pdm_status, there will be an entry for "es_ebl" or the "Search Deamon" process. For conventional configurations, this process is located on the Primary SDM Server. For Advance Availability, the process is on both the Background and the App server running xFlow. Note that there is a known cosmetic spelling error for "Deamon". The "es_ebl" process will connect to Search Server to pass on content from SDM.
How this works is when an entry in SDM is recorded that needs to be passed on to Search Server, the "es_ebl" process will write an entry to the backend database table "es_events", then a "Push" is done to write the content over to the appropriate index in Search Server. Once the push to Search Server is completed, the corresponding entry in the database table "es_events" is deleted. Normally, the "es_events" table is empty, unless there is data to be pushed over to ElasticSearch. Once it is pushed to SS, the "es_events" table entries is deleted.
Log file of interest is the esEvents.log in the SDM log directory. One should examine this log for troubleshooting concerns involving ticket data from SDM to be transferred across to Search Server. To turn up the logging, one must update the esLog4j.properties file in NX_ROOT\site\cfg to debug levels, then run "pdm_bounce es_ebl" to restart the SDM "Search Deamon" process.
Service Catalog/Search Server Interaction
Within Catalog, the interaction is simpler. Catalog will make a direct connection to the Search Server to push its contents over to the appropriate index stored in Search Server.
Log file of interest is the Catalog view.log. One should examine this log for troubleshooting concerns involving request data and offering data from Catalog to be transferred across to Search Server. There will be entries for the Search Server interactions if one searches for "ESClient". View.log would need to have debugging enabled to view the content.
xFlow and Service Point/Search Server Interaction
xFlow and Service Point will utilise the Search Microservice defined in xFlow to interface and obtain materials from Search Server. One may consider turning up logging on the Search Microservice to view the content and materials that xFlow and Service Point obtains for a given transaction between its Search Microservice and the Search Server. Logs of interest will be the searchMS_debug.log and its derivative logs in the xFlow install directory.
Direct Search Server examination
The most common method to view the contents of Search Server's indexes is to utilise the Chrome plugin "ElasticSearch Head". A common activity is to use this functionalty to directly examine a given index stored in Search Server to determine its contents and accuracy when SDM or Catalog writes to the indexes and xFlow or Service Point reads from the indexes.
Information on the plugin is available here:
Once the plugin has been installed on your Chrome browser instance, you can access from this location:
Enter the specific Search Server address and click Connect. You can view and manipulate the indices that have been defined in Search Server:
One can also use REST Web Services Utility such as Postman to view content from Search Server;
Go into Postman and enter the following URL: http://[NAME OF SEARCH SERVER HOST]:9012/_alias
The following shows a sample result, where Search Server does not have its indexes established yet:
After the indexes are created by running pdm_es_initial_load.bat from the Search Server's default location of C:\Program Files\CA\SC\CASearchServer\search\bin, the same action in Postman will show:
The index names that appear will contain an integer value, which represents the date stamp of the index in UNIX time. The above index circled in red, "sdm_index_cnt_1686248460" has a UNIX time of 1686248460 or Thu Jun 08 2023 18:21:00 GMT+0000.
Additional Information
The following articles discuss how to enable tracing for the related Service Management components as well as re-do indexes in Search Server:
KB Article 12894: Enable CA Service Catalog's TRACE (or DEBUG) level for diagnostic purposes
KB Article 45002: Turning on logging for xFlow
KB Article 189262: How to recreate indexes on CA Search Server?
KB Article 267707: Manual rebuild of Search Server indexes
The following is a selection of KB Articles which discuss various issues that have arisen as a result of Service Point failures due to Search Server functionality needing to be addressed.
KB Article 205022: Unable to locate Service catalog offering under service point.
KB Article 267599: Search Server indexes in a cluster are not being generated
KB Article 220855: Service Point is not updating service offerings from Catalog
KB Article 194399: After upgrading Service Management from 17.1 to 17.2, in Service Point no Catalog offerings are displayed
KB Article 254742: Service Catalog Offerings Are Not Showing in Service Point
KB Article 215408: Service Point Mobile App and Service Catalog offerings
KB Article 222009: Service Point does not load Service Catalog Offerings after clicking on More button
KB Article 190719: Service Point does not display Catalog offerings and HTTP 400 Bad Request error appears in F12 Console tab
KB Article 99907: Catalog offerings and data sources not loading for certain users in Service Point
KB Article 265266: Service Point does not load Sevice Catalog offerings when using SAML Authentication
KB Article 141743: Service Point does not show/find Catalog offerings for Employee role only and Error "Error fetching permissions for" appears in searchMS.log
KB Article 199178: Knowledge Search is not working for Service Catalog Featured Items from Service Point
KB Article 240501: Search bar does not find anything when searching
Feedback
