CPU utilization alarm due to a scheduled RAID check

Products

• XOS

Issue/Introduction

On XOS 9.5.x and later, the customer may experience periodic minor CPU utilization peaks on a CPM/APM with disks in a RAID configuration.The following can be observed in the logs (including the timing, i.e. starting every Sunday at 4:22 am):

Aug 5 04:22:01 CBS kernel: md: syncing RAID array md9 

Aug 5 04:22:01 CBS kernel: md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. 

Aug 5 04:22:01 CBS kernel: md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. 

...

Aug 5 04:23:41 CBS cbshmonitord: [N] Violation (s=1, alarm) occurred 3 times: module:13, item:3503 (H_ID_PROC_HI_CORE_UTIL), time:"Sun Aug 5 04:23:21 2012", value: 83, norm:0-80, minor:0-90, major:0-99 

Aug 5 04:23:42 CBS cbsalarmlogrd: AlarmID 537 | Sun Aug 5 04:23:41 2012 | minor | cp1 | cpuCoreUtilizationExceeded | CPU utilization core: 1 

Aug 5 04:23:42 CBS cbsalarmlogrd: AlarmID 538 | Sun Aug 5 04:23:41 2012 | info | system | systemAlarm | New system alarm level (minor) 

Aug 5 04:23:43 CBS cbshmonitord: [I] chassis fault counters: 1 0 0 

Aug 5 04:23:44 CBS cbshmonitord: [N] Violation (s=1, no alarm) occurred: module:13, item:2906 (H_ID_MINOR_LED_ON), time:[1344108224]"Sun Aug 5 04:23:44 2012" 

Aug 5 04:23:46 CBS kernel: md: md10: sync done.

Cause

Goal:

To explain this behavior.

Resolution

The higher CPU utilization is caused by a higher disk activity due to a scheduled Cron job checking the health of the RAID. It is run weekly on Sunday at 4:22 am (as per /etc/crontab). Its purpose is to ensure the consistency of the RAID and is recommended to be left enabled. Higher CPU/disk activity during the RAID check will have only a minimal impact on the overall performance of the box.

In a case where the customer explicitly wishes to have this RAID health check turned off, the file /etc/sysconfig/raid-check on the relevant CPM/APM needs to be edited as follows:

Locate the following line:
ENABLED=yes

And change it to:
ENABLED=no

This change will take effect immediately after saving the file.

Workaround

N/A