Have a prod cluster that is down hard and will not start... MOM is fine. Collectors will not start....
Example: IntroscopeEnterpriseManager.log states:
[INFO] [main] [Manager] Available processors is 6
[ERROR] [main] [Manager] The EM failed to start. com.wily.introscope.spec.metric.BadlyFormedNameException: "Backends|yyydb on xxx (DB2 DB)|SQL|Prepared|Query|" is not a legal Metric name.
[INFO] [main] [Manager] Shutting down the Isengard server
[INFO] [main] [Manager] Orderly shutdown complete.
******************************
Tried looking for the data with SmartstoreTools:
./SmartStorTools.sh test_regex -metrics ".*Backends|yyydb on xxx (DB2 DB)|SQL|.*" -src /CAAPMDATA/Introscope/data > out.txt
Provided the following error and stopped:
log4j:WARN No appenders could be found for logger (org.springframework.osgi.extender.internal.activator.ContextLoaderListener).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
com.wily.introscope.server.enterprise.entity.meta.MetadataException: com.wily.introscope.spec.metric.BadlyFormedNameException: Metric_Name_Is_Badly_Formed
at com.wily.introscope.server.enterprise.entity.meta.MetadataStoreSystem.getMatchingItems(MetadataStoreSystem.java:1127)
at com.wily.introscope.em.smartstor.tools.fileapps.RegexTester.printMatchingMetrics(RegexTester.java:265)
at com.wily.introscope.em.smartstor.tools.fileapps.RegexTester.main(RegexTester.java:148)
at com.wily.introscope.em.smartstor.tools.fileapps.RegexTester.main(RegexTester.java:71)
at com.wily.introscope.em.smartstor.tools.SmartStorTools.main(SmartStorTools.java:77)
at com.wily.introscope.eclipseapp.SmartStorTools.start(SmartStorTools.java:11)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:386)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
at org.eclipse.equinox.launcher.Main.main(Main.java:1212)
Caused by: com.wily.introscope.spec.metric.BadlyFormedNameException: Metric_Name_Is_Badly_Formed
at com.wily.introscope.spec.metric.AgentMetricPrefixParser.validateSegment(AgentMetricPrefixParser.java:118)
at com.wily.introscope.spec.metric.AgentMetricPrefixParser.validate(AgentMetricPrefixParser.java:104)
at com.wily.introscope.spec.metric.AgentMetricPrefixParser.<init>(AgentMetricPrefixParser.java:33)
at com.wily.introscope.spec.metric.AgentMetricPrefix.<init>(AgentMetricPrefix.java:176)
at com.wily.introscope.spec.metric.AgentMetricPrefix.<init>(AgentMetricPrefix.java:174)
at com.wily.introscope.spec.metric.AgentMetricPrefix$Factory.create(AgentMetricPrefix.java:48)
at com.wily.introscope.server.enterprise.entity.meta.cache.AgentMetricPrefixCache.getOrAddToCache(AgentMetricPrefixCache.java:97)
at com.wily.introscope.spec.metric.AgentMetricPrefix.getFromCache(AgentMetricPrefix.java:340)
at com.wily.introscope.spec.metric.AgentMetricPrefix.getAgentMetricPrefix(AgentMetricPrefix.java:369)
at com.wily.introscope.spec.metric.AgentMetric.<init>(AgentMetric.java:455)
at com.wily.introscope.server.enterprise.entity.meta.cache.AgentMetricCache.lookupMetricWithTypeCheck(AgentMetricCache.java:105)
at com.wily.introscope.spec.metric.AgentMetric.lookupMetric(AgentMetric.java:587)
at com.wily.introscope.spec.metric.AgentMetric.getAgentMetricFromCache(AgentMetric.java:616)
at com.wily.introscope.spec.metric.AgentMetric.getAgentMetric(AgentMetric.java:798)
at com.wily.introscope.server.enterprise.entity.meta.table.TableTreeToMetric.getAgentMetric(TableTreeToMetric.java:233)
at com.wily.introscope.server.enterprise.entity.meta.MetadataStoreSystem$14.data(MetadataStoreSystem.java:1104)
at com.wily.introscope.server.enterprise.entity.rocksdb.RocksDBSystem.iterate(RocksDBSystem.java:157)
at com.wily.introscope.server.enterprise.entity.rocksdb.RocksDBSystem.iterate(RocksDBSystem.java:131)
at com.wily.introscope.server.enterprise.entity.meta.MetadataStoreSystem.getMatchingItems(MetadataStoreSystem.java:1088)
... 18 more
Release : 10.7.0
Component : APM Agents