search cancel

Archive Manager is going down - seeing connection issues

book

Article ID: 252494

calendar_today

Updated On:

Products

CA Spectrum

Issue/Introduction


Aug 31 00:00:29 ERROR TRACE at ModelArchDBImp.cc(4970): doSqlQuery/mysql_query: Failure executing query:
ALTER TABLE event DROP PARTITION EVENT_31_7_2022 - Error in list of partitions to DROP
Sep 14 23:34:55 ERROR TRACE at ModelArchDBImp.cc(4970): doSqlQuery/mysql_query: Failure executing query:
 SELECT  E.model_h, M.model_name, M.mtype_h, MT.mtype_name, U.user_name, E.type, E.severity, E.vardata_string FROM model_type as MT, model as M, event as E, user_def as U WHERE MT.mtype_h = M.mtype_h AND M.model_h = E.model_h AND E.user_key = U.user_key AND E.utime = 1663212889 AND E.counter = 162712341 AND E.clk_seq = 557 AND E.version = 1 AND E.node_id = '\0PV–¬' - Lost connection to MySQL server during query
Sep 14 23:35:09 ERROR TRACE at ModelArchDBImp.cc(4970): doSqlQuery/mysql_query: Failure executing query:
 SELECT  E.model_h, M.model_name, M.mtype_h, MT.mtype_name, U.user_name, E.type, E.severity, E.vardata_string FROM model_type as MT, model as M, event as E, user_def as U WHERE MT.mtype_h = M.mtype_h AND M.model_h = E.model_h AND E.user_key = U.user_key AND E.utime = 1663212881 AND E.counter = 162711766 AND E.clk_seq = 557 AND E.version = 1 AND E.node_id = '\0PV–¬' - Lost connection to MySQL server during query
Sep 14 23:38:29 ERROR TRACE at ModelArchDBImp.cc(4970): doSqlQuery/mysql_query: Failure executing query:
 SELECT  E.model_h, M.model_name, M.mtype_h, MT.mtype_name, U.user_name, E.type, E.severity, E.vardata_string FROM model_type as MT, model as M, event as E, user_def as U WHERE MT.mtype_h = M.mtype_h AND M.model_h = E.model_h AND E.user_key = U.user_key AND E.utime = 1663213094 AND E.counter = 162729304 AND E.clk_seq = 557 AND E.version = 1 AND E.node_id = '\0PV–¬' - Lost connection to MySQL server during query
Sep 14 23:38:30 ERROR TRACE at ModelArchDBImp.cc(4970): doSqlQuery/mysql_query: Failure executing query:
INSERT IGNORE INTO event(model_h, utime, counter, clk_seq, version, node_id, user_key, type, severity, vardata_string) VALUES(28645013,1663213090,162729001,557,1,'\0PV–¬',1,2162715,50,'\0\0\0\0\0\0\0\0\0\0\0\0NHRP\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0PAKERROR\0\0\0\0\0\0Ò\0\0\0Received Error Indication from 10.166.41.2, code: protocol generic error(7), (trigger src: 10.166.41.239 (nbma: 172.16.50.57) dst: 10.166.40.2), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 6C 2B 6C 00 34 \0\0\0\0\0\0,»}Ð#\0\0\0\0,»}') - Lost connection to MySQL server during query
ModelArchDb ERROR at DDMDbIf2.cc(952): 
Generic Error: 1, commitTransaction/doSqlQuery/mysql_query: Failure executing query:
INSERT IGNORE INTO event(model_h, utime, counter, clk_seq, version, node_id, user_key, type, severity, vardata_string) VALUES(28645013,1663213090,162729001,557,1,'\0PV–¬',1,2162715,50,'\0\0\0\0\0\0\0\0\0\0\0\0NHRP\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0PAKERROR\0\0\0\0\0\0Ò\0\0\0Received Error Indication from 10.166.41.2, code: protocol generic error(7), (trigger src: 10.166.41.239 (nbma: 172.16.50.57) dst: 10.166.40.2), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 6C 2B 6C 00 34 \0\0\0\0\0\0,»}Ð#\0\0\0\0,»}')
Native Error: 2013, Lost connection to MySQL server during query
Sep 14 23:38:30 ERROR TRACE at DDMDbIf.cc(2930): Error attempting to end transaction.
ModelArchDb ERROR at DDMDbIf.cc(2940): 
Generic Error: 1, commitTransaction/doSqlQuery/mysql_query: Failure executing query:
INSERT IGNORE INTO event(model_h, utime, counter, clk_seq, version, node_id, user_key, type, severity, vardata_string) VALUES(28645013,1663213090,162729001,557,1,'\0PV–¬',1,2162715,50,'\0\0\0\0\0\0\0\0\0\0\0\0NHRP\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0PAKERROR\0\0\0\0\0\0Ò\0\0\0Received Error Indication from 10.166.41.2, code: protocol generic error(7), (trigger src: 10.166.41.239 (nbma: 172.16.50.57) dst: 10.166.40.2), offset: 0, data: 00 01 08 00 00 00 00 00 00 FF 00 6C 2B 6C 00 34 \0\0\0\0\0\0,»}Ð#\0\0\0\0,»}')
Native Error: 2013, Lost connection to MySQL server during query
Sep 14 23:38:30 ERROR TRACE at DDMDbIf.cc(2942): Aborting database transaction.
Sep 14 23:38:30 ERROR TRACE at CsEventMgr.cc(1758): Event Log Manager: Error attempting to log event to database.
Sep 14 23:38:43 ERROR TRACE at CsEventMgr.cc(1779): Fatal Event Log Manager Error.  Scheduling ArchMgr shutdown.
Sep 14 23:38:43 : Closing all client connections...
Sep 14 23:38:43 : Stopping CORBA communication...
Sep 14 23:39:02 : Waiting for 1345 pending database writes to complete...
Sep 14 23:39:05 : Closing database
Sep 14 23:39:05 : ArchMgr is shutting down...
Sep 14 23:39:05 : ArchMgr has successfully shut down.

 

 

Environment

Release : 21.2

Resolution

In you my-spectrum.cnf add:

net_read_timeout=60

connect_timeout=30

Also, try to reduce the incoming event volume.  Increased volume is the ultimate problem here.