"BUG: soft lockup - CPU#<id> stuck for 10s!" message seen in the logs, APM crashed and restarted

Article Id: 168763
Status: Published
Updated On: 13-05-2017 12:00
Legacy Id: TECH245466
Products:
XOS APM
Issue/Introduction:

A VAP member rebooted and produced a crash output containing a line similar to the following:

kernel: BUG: soft lockup - CPU#5 stuck for 10s! [fwk10_0:17177] 

Cause:

The reboot and error result when a prolonged soft lockup occurs. The cause of the soft lockup itself needs to be investigated depending on the crash output.

Resolution:

In XOS 9.6.8+, XOS 9.7.3+, and XOS 10.0.0+, XOS can detect a prolonged CPU soft lockup condition and trigger an automatic restart of the APM,

From the Release Notes: 

ID 102504 - Enhanced XOS to detect a prolonged CPU soft lockup condition, and automatically cause the APM to restart, (activating the XOS redundancy features) when the lockup condition is recognized (Red Hat ID 445422).


To disable this functionality, do one of the following:

  1. Run "echo 0 > /proc/sys/kernel/softlockup_panic" This change does not survive a VAP member reboot.
  2. Change sysctl.conf (kernel.softlockup_panic = 0) on the VAP member to make the change permanent.

Workaround

N/A