Spectrum mysql server does not remain running after patching RHEL OS

Products

• Spectrum

Issue/Introduction

MYSQL.OUT shows the following

2023-11-30T01:01:35.949598Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-11-30T01:01:35.958325Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_9_11_2023.ibd'
2023-11-30T01:01:35.958392Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_10_11_2023.ibd'
2023-11-30T01:01:35.958423Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_11_11_2023.ibd'
2023-11-30T01:01:35.958448Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_12_11_2023.ibd'
2023-11-30T01:01:35.958468Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_13_11_2023.ibd'
2023-11-30T01:01:35.958486Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_14_11_2023.ibd'
2023-11-30T01:01:35.958511Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/model.ibd'
2023-11-30T01:01:35.958531Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/model_type.ibd'
2023-11-30T01:01:35.958556Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_24_11_2023.ibd'
2023-11-30T01:01:35.958600Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_30_11_2023.ibd'
2023-11-30T01:01:35.958641Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_29_10_2023.ibd'
2023-11-30T01:01:35.958674Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_31_10_2023.ibd'
2023-11-30T01:01:35.958695Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_30_10_2023.ibd'
2023-11-30T01:01:35.958713Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_26_10_2023.ibd'
2023-11-30T01:01:35.958737Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_16_10_2023.ibd'
2023-11-30T01:01:35.958756Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_19_10_2023.ibd'
2023-11-30T01:01:35.958778Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_23_10_2023.ibd'
2023-11-30T01:01:35.958799Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_5_11_2023.ibd'
2023-11-30T01:01:35.958817Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_2_11_2023.ibd'
2023-11-30T01:01:35.958840Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/attribute.ibd'
2023-11-30T01:01:35.958860Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_28_11_2023.ibd'
2023-11-30T01:01:35.958878Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_8_11_2023.ibd'
2023-11-30T01:01:35.958902Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_25_10_2023.ibd'
2023-11-30T01:01:35.958921Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_28_10_2023.ibd'
2023-11-30T01:01:35.958955Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/statistic_ul64.ibd'
2023-11-30T01:01:35.958980Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/user_def.ibd'
2023-11-30T01:01:35.959006Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_18_11_2023.ibd'
2023-11-30T01:01:35.959025Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_21_11_2023.ibd'
2023-11-30T01:01:35.959042Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_26_11_2023.ibd'
2023-11-30T01:01:35.959068Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_7_11_2023.ibd'
2023-11-30T01:01:35.959085Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_24_10_2023.ibd'
2023-11-30T01:01:35.959109Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/landscape.ibd'
2023-11-30T01:01:35.959129Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/model_name_extension.ibd'
2023-11-30T01:01:35.959147Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/statistic.ibd'
2023-11-30T01:01:35.959173Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/statistic_string.ibd'
2023-11-30T01:01:35.959191Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_16_11_2023.ibd'
2023-11-30T01:01:35.959215Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_19_11_2023.ibd'
2023-11-30T01:01:35.959234Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_22_11_2023.ibd'
2023-11-30T01:01:35.959275Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_27_11_2023.ibd'
2023-11-30T01:01:35.959295Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_17_10_2023.ibd'
2023-11-30T01:01:35.959312Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_21_10_2023.ibd'
2023-11-30T01:01:35.959336Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_22_10_2023.ibd'
2023-11-30T01:01:35.959356Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_3_11_2023.ibd'
2023-11-30T01:01:35.959373Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_4_11_2023.ibd'
2023-11-30T01:01:35.959398Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_27_10_2023.ibd'
2023-11-30T01:01:35.959417Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_20_10_2023.ibd'
2023-11-30T01:01:35.959434Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_6_11_2023.ibd'
2023-11-30T01:01:35.959451Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_18_10_2023.ibd'
2023-11-30T01:01:35.959468Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_15_11_2023.ibd'
2023-11-30T01:01:35.959485Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/partitionlog.ibd'
2023-11-30T01:01:35.959503Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_17_11_2023.ibd'
2023-11-30T01:01:35.959520Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_20_11_2023.ibd'
2023-11-30T01:01:35.959537Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_23_11_2023.ibd'
2023-11-30T01:01:35.959573Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_29_11_2023.ibd'
2023-11-30T01:01:35.959616Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_1_11_2023.ibd'
2023-11-30T01:01:35.959657Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/db_descriptor.ibd'
2023-11-30T01:01:35.959691Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/model_attribute.ibd'
2023-11-30T01:01:35.959712Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/oid_suffix.ibd'
2023-11-30T01:01:35.959730Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './ddmdb/event#p#event_25_11_2023.ibd'
2023-11-30T01:01:35.959902Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './mysql/backup_progress.ibd'
2023-11-30T01:01:35.959933Z 1 [Warning] [MY-012197] [InnoDB] Unable to open './mysql/backup_history.ibd'
2023-11-30T01:01:36.286495Z 1 [ERROR] [MY-012884] [InnoDB] ./#innodb_redo/#ib_redo153 can't be opened in read-write mode.
2023-11-30T01:01:36.286607Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
2023-11-30T01:01:36.689820Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2023-11-30T01:01:36.690290Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2023-11-30T01:01:36.690331Z 0 [ERROR] [MY-010119] [Server] Aborting
2023-11-30T01:01:36.691839Z 0 [System] [MY-010910] [Server] /app/CA/Spectrum/mysql/bin//mysqld: Shutdown complete (mysqld 8.0.33-commercial)  MySQL Enterprise Server - Commercial.

Cause

• Permissions issue
• mysql files owned by spectrum install owner were changed to root

 

Resolution

• Check all files under SPECROOT/mysql/data/ddmdb are owned by spectrum install owner account, if not change file ownership using chown

Example:  caadmin:caadmin * 

There may be other files that are also affected.   

Review the MYSQL.OUT log for any other files that cannot be opened and repeat the same chown command against those files.