Fatal errors in spectrumgtw probe
Article ID: 121376
Updated On:
Products
DX Unified Infrastructure Management (Nimsoft / UIM)
Issue/Introduction
Seeing a lot of fatal errors in the spectrumgtw probe. Often this leads to alarms not showing in spectrum -there is still a connection however no new alarms arrive
Oct 01 2018 12:22:39,164 [attach_clientsession] ERROR com.nimsoft.nimbus.NimSubscribe - Not able to callback for hubpost. Reason is: java.lang.reflect.InvocationTargetException Oct 01 2018 12:22:39,170 [attach_clientsession] FATAL com.nimsoft.nimbus.NimSubscribe - pdsdata: user_tag_1=[CA UIM], , charset=windows-1252, pri=1, subject=udm_inventory, origin=VPPrimary_hub, hop=0, prid=discovery_server, source=10.217.41.32, nimts=1538360559, robot=PMWIM017, nimid=SH80463471-16964, tz_offset=-36000, md5sum= , hop0=VPPrimary_hub, domain=VictoriaPolice_domain, qsize=1 udata: mimeType=application/x-deflate, graph=xÚíW[o£Fþ+OdÐÃ৚[b5‰Ý8I[U+kãd´ÀPÀÙºÑþ÷0ñ’8Êz7[E‘òdqæÜÏ÷�ßi±ÊŠTÔâ¨äÅ�6ªËµjóøFdBÝÁq¾’×�XÉ\Ö Oct 01 2018 12:22:39,180 [attach_clientsession] ERROR com.nimsoft.nimbus.NimSubscribe - java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.nimsoft.nimbus.NimSubscribe.callbackProxy(NimSubscribe.java:364) at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.performCallback(NimServerSession.java:216) at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.handleMessage(NimServerSession.java:169) at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.run(NimServerSession.java:123) Caused by: org.mapdb.DBException$FileLocked: File is already opened and is locked: cache\index at org.mapdb.volume.Volume.lockFile(Volume.java:495) at org.mapdb.volume.RandomAccessFileVol.(RandomAccessFileVol.java:52) at org.mapdb.volume.RandomAccessFileVol$1.makeVolume(RandomAccessFileVol.java:26) at org.mapdb.StoreWAL$realVolume$1.invoke(StoreWAL.kt:75) at org.mapdb.StoreWAL$realVolume$1.invoke(StoreWAL.kt:18) at org.mapdb.StoreWAL.(StoreWAL.kt:74) at org.mapdb.StoreWAL$Companion.make(StoreWAL.kt:56) at org.mapdb.StoreWAL$Companion.make$default(StoreWAL.kt:55) at org.mapdb.DBMaker$Maker.make(DBMaker.kt:464) at com.uim.pusher.DeviceIndex.(DeviceIndex.java:29) at com.uim.pusher.GraphTransformer.(GraphTransformer.java:137) at com.uim.pusher.GraphTransformer.(GraphTransformer.java:43) at com.uim.pusher.GraphTransformer$Builder.build(GraphTransformer.java:56) at com.uim.pusher.UdmInventoryPdsHandler.handleEachPds(UdmInventoryPdsHandler.java:132) at com.uim.pusher.UdmInventoryPdsHandler.handlePds(UdmInventoryPdsHandler.java:107) at com.nimsoft.probe.gateway.spectrumgtw.SpectrumGtwProbe.hubpostCallback(SpectrumGtwProbe.java:445) ... 10 more Caused by: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1108) at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155) at org.mapdb.volume.Volume.lockFile(Volume.java:490) ... 25 more
Oct 01 2018 12:22:39,164 [attach_clientsession] ERROR com.nimsoft.nimbus.NimSubscribe - Not able to callback for hubpost. Reason is: java.lang.reflect.InvocationTargetException Oct 01 2018 12:22:39,170 [attach_clientsession] FATAL com.nimsoft.nimbus.NimSubscribe - pdsdata: user_tag_1=[CA UIM], , charset=windows-1252, pri=1, subject=udm_inventory, origin=VPPrimary_hub, hop=0, prid=discovery_server, source=10.217.41.32, nimts=1538360559, robot=PMWIM017, nimid=SH80463471-16964, tz_offset=-36000, md5sum= , hop0=VPPrimary_hub, domain=VictoriaPolice_domain, qsize=1 udata: mimeType=application/x-deflate, graph=xÚíW[o£Fþ+OdÐÃ৚[b5‰Ý8I[U+kãd´ÀPÀÙºÑþ÷0ñ’8Êz7[E‘òdqæÜÏ÷�ßi±ÊŠTÔâ¨äÅ�6ªËµjóøFdBÝÁq¾’×�XÉ\Ö Oct 01 2018 12:22:39,180 [attach_clientsession] ERROR com.nimsoft.nimbus.NimSubscribe - java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.nimsoft.nimbus.NimSubscribe.callbackProxy(NimSubscribe.java:364) at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.performCallback(NimServerSession.java:216) at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.handleMessage(NimServerSession.java:169) at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.run(NimServerSession.java:123) Caused by: org.mapdb.DBException$FileLocked: File is already opened and is locked: cache\index at org.mapdb.volume.Volume.lockFile(Volume.java:495) at org.mapdb.volume.RandomAccessFileVol.(RandomAccessFileVol.java:52) at org.mapdb.volume.RandomAccessFileVol$1.makeVolume(RandomAccessFileVol.java:26) at org.mapdb.StoreWAL$realVolume$1.invoke(StoreWAL.kt:75) at org.mapdb.StoreWAL$realVolume$1.invoke(StoreWAL.kt:18) at org.mapdb.StoreWAL.(StoreWAL.kt:74) at org.mapdb.StoreWAL$Companion.make(StoreWAL.kt:56) at org.mapdb.StoreWAL$Companion.make$default(StoreWAL.kt:55) at org.mapdb.DBMaker$Maker.make(DBMaker.kt:464) at com.uim.pusher.DeviceIndex.(DeviceIndex.java:29) at com.uim.pusher.GraphTransformer.(GraphTransformer.java:137) at com.uim.pusher.GraphTransformer.(GraphTransformer.java:43) at com.uim.pusher.GraphTransformer$Builder.build(GraphTransformer.java:56) at com.uim.pusher.UdmInventoryPdsHandler.handleEachPds(UdmInventoryPdsHandler.java:132) at com.uim.pusher.UdmInventoryPdsHandler.handlePds(UdmInventoryPdsHandler.java:107) at com.nimsoft.probe.gateway.spectrumgtw.SpectrumGtwProbe.hubpostCallback(SpectrumGtwProbe.java:445) ... 10 more Caused by: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1108) at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155) at org.mapdb.volume.Volume.lockFile(Volume.java:490) ... 25 more
Environment
Spectrumgtw-8.65-HF2
Spectrum UIM service-8.65
NAS: 9.03
EMS: 10.17
Maintenance mode: 8.53
vm ware: 7.11
Spectrum 10.2.3: Legacy integration.
UIM Device: 1933
VMS: 1418
ESX:169:
Datacenters: 5
Clusters: 15
Spectrum UIM service-8.65
NAS: 9.03
EMS: 10.17
Maintenance mode: 8.53
vm ware: 7.11
Spectrum 10.2.3: Legacy integration.
UIM Device: 1933
VMS: 1418
ESX:169:
Datacenters: 5
Clusters: 15
Cause
That cache folder appears to be the issue as size increases very fast
The instability occurs when the cache size grows large
The instability occurs when the cache size grows large
Resolution
If size is grows more, take below workaround steps:
1. Stop the spectrumgtw probe
2. remove all files from Cache folder except *.dat files.
3. start Spectrumgtw probe.
Permanent fix
spectrumgtw-8.6.5-HF7 from UIM hotfix link
1. Stop the spectrumgtw probe
2. remove all files from Cache folder except *.dat files.
3. start Spectrumgtw probe.
Permanent fix
spectrumgtw-8.6.5-HF7 from UIM hotfix link
Additional Information
This may also occur in spectrumgtw 8.67 - please use 8.67HF4 to correct the issue.
Feedback
Yes
No
Powered by
