Fetch from cache has failed with exception

Products

• DX OI SaaS

Issue/Introduction

What means the below logs? Is there any way to fix it?

ERROR AlarmBuilder:317 - sedalerts~~sedalerts - Fetch from cache has failed with exception java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException at com.google.common.cache.LocalCache.loadAll(LocalCache.java:4085) at com.google.common.cache.LocalCache.getAll(LocalCache.java:4036) at com.google.common.cache.LocalCache$LocalLoadingCache.getAll(LocalCache.java:4964) at com.ca.ce.restmon.publish.alarms.AlarmBuilder.getAlarmsFromCacheMatchingAlarmSpec(AlarmBuilder.java:482) at com.ca.ce.restmon.publish.alarms.AlarmBuilder.getAlarmsToBePublishedToOI(AlarmBuilder.java:314) at com.ca.ce.restmon.publish.alarms.AlarmBuilder.buildAndPublishAlarms(AlarmBuilder.java:596) at com.ca.ce.restmon.publish.alarms.AlarmBuilder.buildAndPublishAlarms(AlarmBuilder.java:545) at com.ca.ce.restmon.publish.doi.DoiPublisher.publishAndForgetAlarmData(DoiPublisher.java:158) at com.ca.ce.restmon.handlers.profile.ProfileHandler.checkPublishAlarms(ProfileHandler.java:1168) at com.ca.ce.restmon.handlers.profile.ProfileHandler.lambda$processAndPublishAlarmDataForLog$4(ProfileHandler.java:1113) at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1705) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:588) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:113) at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:240) at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2317) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2283) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2159) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2049) at com.google.common.cache.LocalCache.get(LocalCache.java:3966) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3989) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4950) at com.ca.ce.restmon.app.cache.caches.PublishedAlarmsCache.getAlarmObjectsFortheGivenAlarmSpecs(PublishedAlarmsCache.java:172) at com.ca.ce.restmon.app.cache.caches.PublishedAlarmsCache$AlarmSpecifierPublishedAlarmCache.loadAllValues(PublishedAlarmsCache.java:122) at com.ca.ce.restmon.app.cache.CacheManager$3.loadAll(CacheManager.java:155) at com.google.common.cache.LocalCache.loadAll(LocalCache.java:4073) ... 15 more Caused by: java.util.concurrent.ExecutionException at com.ca.ce.restmon.publish.alarms.api.JarvisGatewayConfiguration.processResponse(JarvisGatewayConfiguration.java:153) at com.ca.ce.restmon.publish.alarms.api.JarvisGatewayConfiguration.sendRequest(JarvisGatewayConfiguration.java:127) at com.ca.ce.restmon.publish.alarms.api.JarvisGatewayConfiguration.queryByProduct(JarvisGatewayConfiguration.java:115) at com.ca.ce.restmon.publish.alarms.AlarmBatchPublisher.queryForAlarmIdsByProduct(AlarmBatchPublisher.java:81) at com.ca.ce.restmon.app.cache.caches.AlarmsByProductCache.getActiveAlarmIdsByProductSpec(AlarmsByProductCache.java:99) at com.ca.ce.restmon.app.cache.caches.AlarmsByProductCache$ProductAlarmsCache.loadValue(AlarmsByProductCache.java:85) at com.ca.ce.restmon.app.cache.caches.AlarmsByProductCache$ProductAlarmsCache.loadValue(AlarmsByProductCache.java:49) at com.ca.ce.restmon.app.cache.CacheManager$3.load(CacheManager.java:147) at com.ca.ce.restmon.app.cache.CacheManager$3.load(CacheManager.java:143) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3533) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2282) ... 24 more

 

Environment

Release : SAAS

Resolution

The issue is because of the longer alarm_unqiue_id used,  which is longer than the supported length by ElasticSearch.

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Thread.java:829) Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://elasticsearch-client.ao-jarvis.svc.cluster.local:9200], URI [/ao_itoa_alarms_all_3*/_search?scroll=10m&typed_keys=true&max_concurrent_shard_requests=5&search_type=query_then_fetch&batched_reduce_size=512], status line [HTTP/1.1 400 Bad Request] {"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: The length of regex [1023] used in the Regexp Query request has exceeded the allowed maximum of [1000]. This maximum can be set by changing the [index.max_regex_length] index level setting.","index_uuid":"jGTc-vo2TwCz6veWEZhsnQ","index":"ao_itoa_alarms_all_3_34"},{"type":"query_shard_exception","reason":"failed to create query: The length of regex [1023] used in the Regexp Query request has exceeded the allowed maximum of [1000]. This maximum can be set by changing the [index.max_regex_length] index level setting.","index_uuid":"pFQuyETRRvGZSQvHDAY-hA","index":"ao_itoa_alarms_all_3_35"},{"type":"query_shard_exception","reason":"failed to create query: The length of regex [1023] used in the Regexp Query request has exceeded the allowed maximum of [1000]. This maximum can be set by changing the [index.max_regex_length] index level setting.","index_uuid":"dDvpGhT8QXObo56_jEAuvw","index":"ao_itoa_alarms_all_3_36"},{"type":"query_shard_exception","reason":"failed to create query: The length of regex [1023] used in the Regexp Query request has exceeded the allowed maximum of [1000]. This maximum can be set by changing the [index.max_reg