Sudden degraded performance on one gateway in a cluster of 2
Article ID: 254176
Updated On:
Products
Issue/Introduction
We are recently observing performance issues with one of 2 gateways in our cluster.
We are logging response times and they only seem to be high on a single node.
We restarted the services and also rebooted both servers and are still having the issue. I ran a data collection and have attached it here for your review.
Environment
Release : 10.0
Cause
The Primary SSG DB used by the Gateway is having an issue. There is a known bug causing MySQL to crash.
Stack analysis has threads blocked in MySQL, the MySQL log has the following crash logged
2022-10-28T19:23:36.593814Z 5 [ERROR] [MY-010584] [Repl] Slave I/O for channel '': error connecting to master '[email protected]:3307' - retry-time: 100 retries: 4 message: Can't connect to MySQL server on 'HOSTNAME' (111), Error_code: MY-002003
19:25:16 UTC - mysqld got signal 11 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x7f0b380008c0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f0c08b309c8 thread_stack 0x46000
/usr/sbin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x21e109d]
/usr/sbin/mysqld(handle_fatal_signal+0x313) [0x10096b3]
/lib64/libpthread.so.0(+0xf630) [0x7f0c1ec0d630]
/usr/sbin/mysqld(mysql_compress_context_init(mysql_compress_context*, enum_compression_algorithm, unsigned int)+0) [0x21d9000]
/usr/sbin/mysqld() [0xfe9e05]
Resolution
Recommend updating to the latest Platform patch Layer7_API_PlatformUpdate_64bit_v10.X-CentOS-2022-09-22.zip
Link:
The current version you are running is: mysqld 8.0.23-commercial
Newer patches: '8.0.30-commercial
Feedback
