Discovery server queue not working as expected

book

Article ID: 207256

calendar_today

Updated On:

Products

DX Infrastructure Management

Issue/Introduction

We found that discovery server is not working as expected. Hub Queue is filling up but not sent during some time periods, and also we had to delete java core dumps generated.

Environment

Release : 20.3

Component : UIM - DISCOVERY_SERVER

Resolution

Please do the following:

1. Allocate more (e.g., double) Java Memory for the discovery_server probe open the raw configuration with Shift - right mouse click on the probe in IM.

select startup -> opt


update the following values or add 2 GB to the min and to the max above your current settings:

java_opts = -server -XX:ErrorFile=./hs_err_pid.log
java_mem_max = -Xmx6144m
java_mem_init = -Xms4096m

2. Rebuild tables indexes using a daily job that runs off-hours (if you do not have partitioning enabled)

These are the key tables with indexes that require defragmentation (DAILY)

Ask your DBA to set up a job to run this DAILY (off-hours) index defrag job.

ALTER INDEX ALL ON CM_COMPUTER_SYSTEM REBUILD;
ALTER INDEX ALL ON CM_DEVICE REBUILD;
ALTER INDEX ALL ON CM_COMPUTER_SYSTEM_ATTR REBUILD;
ALTER INDEX ALL ON CM_DEVICE_ATTRIBUTE REBUILD;
ALTER INDEX ALL ON CM_CONFIGURATION_ITEM REBUILD;
ALTER INDEX ALL ON CM_CONFIGURATION_ITEM_METRIC REBUILD;
ALTER INDEX ALL ON CM_CONFIGURATION_ITEM_DEFINITION REBUILD;
ALTER INDEX ALL ON CM_CONFIGURATION_ITEM_METRIC_DEFINITION REBUILD;
ALTER INDEX ALL ON CM_NIMBUS_ROBOT REBUILD;
ALTER INDEX ALL ON CM_DEVICE REBUILD;
ALTER INDEX ALL ON CM_COMPUTER_SYSTEM_ORIGIN REBUILD;
ALTER INDEX ALL ON CM_CONFIGURATION_ITEM_ATTRIBUTE REBUILD;
ALTER INDEX ALL ON CM_RELATIONSHIP_CI_CI REBUILD;
ALTER INDEX ALL ON CM_RELATIONSHIP_CI_CS REBUILD;
ALTER INDEX ALL ON CM_RELATIONSHIP_CS_CI REBUILD;
ALTER INDEX ALL ON CM_DISCOVERY_NETWORK REBUILD;
ALTER INDEX ALL ON S_QOS_DATA REBUILD;
ALTER INDEX ALL ON S_QOS_DEFINITION REBUILD;
ALTER INDEX ALL ON S_QOS_SNAPSHOT REBUILD;
ALTER INDEX ALL ON NAS_TRANSACTION_SUMMARY REBUILD;
ALTER INDEX ALL ON NAS_ALARMS REBUILD;

Next, in udm_manager.cfg:

1.)In the " Raw Configure" setup, add the following key under the " Setup" section: " schema_transact_retries" and set the value to 5000.

2.) Also change the "schema_connection_timeout_minutes" value to 90


- Also add the following key within the setup section in the discovery_server.cfg

-> section

nis_cache_update_interval_secs = 3600


This will increase the discovery scan time so that it has more time to do its work.

Also check your udm_manager settings,see setup->datomic section:

memory_index_max = 516m
memory_index_threshold = 64m
object_cache_max = 256m
heartbeat_interval_msec = 30000

Note that any time the vmware or another storage probe is restarted, larger graphs will make it to the queue and the queue can then fall behind once again but should only fall behind for a while then catch up.