Out of memory event causes APM kernel panic when running Check Point Security Gateway R70/R71/R75 kernels or Check Point VSX NGX R67/R68 kernels.A kernel panic and subsequent reboot of an APM, generates a cbsoops with a backtrace EIP similar to the following (in the CPM’s /var/log/messages file):

•    [crashes-cbs_pci_dp8650]__netag_put_data+50c
•    [cbs_pci_dp8650]__xskb_conv_to_xctrl+238

The following messages may be logged during this event and are indicative of this problem.

•    KERNEL: assertion (flags & MSG_PEEK) failed at net/ipv4/tcp.c

These kernel panic crashes may occur with regular frequency.

A software issue has been identified in the Linux TCP/IP stack that can lead to an out of memory event and cause an APM kernel panic in the Check Point Security Gateway R70/R71/R75 kernels or the Check Point VSX NGX R67/R68 kernels.

All future XOS maintenance releases will include the fix to prevent the memory resource exhaustion that causes the assertion. 

Specifically, this fix will be incorporated into the following versions to be released in the future: XOS 9.6.1, XOS 9.5.6, XOS 9.0.4, and XOS 8.5.6.  A system running an earlier XOS release must be upgraded to obtain the fix.

Workaround

Changing TCP/IP socket receive buffer queue sizes on the affected APM module(s) will partially mitigate exposure to this Linux kernel issue.  Instructions to adjust the related parameters are below:

• Login into the chassis and from the unix prompt “rsh” to the related APM:
  • rsh <vap-group>_<index>
• To change the parameters dynamically (takes affect immediately but not persistent through reboot):
  • echo "4096 262144 262144" > /proc/sys/net/ipv4/tcp_rmem
• To confirm the parameter change:
  • sysctl -a |grep tcp_rmem
• To make the change persistent through a reboot:
  • vi /etc/sysctl.conf - and add:
  • net.ipv4.tcp_rmem=4096 262144 262144