Gateway unable to start due mysql server do not start -  Error: 30 (Read-only file system)
search cancel

Gateway unable to start due mysql server do not start -  Error: 30 (Read-only file system)

book

Article ID: 389559

calendar_today

Updated On: 03-05-2025

Products

CA API Gateway

Issue/Introduction

The gateway service is unable to start due the MySQL server service not starting. Below is the observed error:

 
[root@gwserver ~]# systemctl status mysql.service


mysql.service - MySQL Enterprise Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2025-02-28 06:21:01 MST; 17s ago
       Docs: man:mysqld(8)
              http://dev.mysql.com/doc/refman/en/using-systemd.html
    Process: 23696 ExecStartPre=/usr/share/mysql-8.0/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
    Process: 23731 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
   Main PID: 23731 (code=exited, status=1/FAILURE)
     Status: "Server shutdown complete"
      Error: 30 (Read-only file system)
        CPU: 301ms

- /opt/SecureSpan/Gateway/node/default/var/logs/ssg_0_0.log  file shows :
 
Feb 28 06:21:01 gwserver systemd[1]: Starting MySQL Enterprise Server...
Feb 28 06:21:01 gwserver systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Feb 28 06:21:01 gwserver systemd[1]: mysql.service: Failed with result 'exit-code'.
Feb 28 06:21:01 gwserver systemd[1]: Failed to start MySQL Enterprise Server.
mysql.service - MySQL Enterprise Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2025-02-28 06:21:01 MST; 17s ago
       Docs: man:mysqld(8)
              http://dev.mysql.com/doc/refman/en/using-systemd.html
    Process: 23696 ExecStartPre=/usr/share/mysql-8.0/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
    Process: 23731 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
    Main PID: 23731 (code=exited, status=1/FAILURE)
    Status: "Server shutdown complete"
    Error: 30 (Read-only file system)
    CPU: 301ms

Environment

CA API Gateway 11.x

Cause

- The filesystem will usually go into read-only mode while the system is running if there is a filesystem consistency issue.
- This is specified in fstab as errors=remount-ro and will occur when a File System access fails or an emergency read-only remount is requested via Alt+SysRq+U
- Also error can be produced because user have set my.cnf with line below :

[mysqld]
  innodb_force_recovery = 1 

 

Resolution

1. Get the drive name with the following command :

[root@gwserver ~]# mount | grep ext4
/dev/mapper/vg01-lv_root on / type ext4 (rw,relatime)
/dev/mapper/vg01-lv_tmp on /tmp type ext4 (rw,nosuid,nodev,relatime)
/dev/mapper/vg01-lv_opt on /opt type ext4 (rw,nodev,relatime)
/dev/mapper/vg01-lv_home on /home type ext4 (rw,nosuid,nodev,relatime)
/dev/mapper/vg01-lv_var on /var type ext4 (rw,nosuid,nodev,relatime)
/dev/mapper/vg01-lv_db on /var/lib/mysql type ext4 (rw,nodev,relatime)
/dev/mapper/vg01-lv_log on /var/log type ext4 (rw,nodev,relatime)
/dev/mapper/vg01-lv_vartmp on /var/tmp type ext4 (rw,nosuid,nodev,relatime)
/dev/mapper/vg01-lv_audit on /var/log/audit type ext4 (rw,nodev,relatime)


2. Unmount the partition ( /tmp partition was unable to be mounted following reboot and on startup - the OS was introduced in maintenance mode)

You can run the command below to force a check of all filesystems:

sudo fsck -Af -M (Don't forget the -M on the command-line)

NOTE: If fsck gets stuck after its version banner:

$ sudo fsck -Af -M
fsck from util-linux 2.20.1

- you may want to try using the EXT4-specific fsck

$ sudo fsck.ext4 -f /dev/sda1
Provided the partition in question /dev/sda1 was an ext4 filesystem.

*** Ran fsck with the appropriate options ( in this case fsck /tmp)

Alternatively, you can run /opt/DCT/.collect.sh -a and check the /home/ssgconfig/ssg-dct-***/os folder for details.

3. Reboot the Gateway server

Additional Information

If Database replication was broken due the error please use the following KB to rebuild replication: Reinitialize Replication in a multi-node cluster