ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

Red Hat Linux 8 server crashed due to a kernel panic which was triggered by the sisips module in DCS agent for Linux

book

Article ID: 237722

calendar_today

Updated On:

Products

Data Center Security Server Advanced

Issue/Introduction

Two different Linux servers unexpectedly halted/crashed on the same day. The initial report in the memory dump that was generated showed the same cause.  The following output (excerpt) shows the perceived.

 

The vmcore shows that server was panicked due to to Unable to handle kernel NULL pointer dereference at 0000000000000000 .- This vmcore shows same symptoms as per prevous kernel crash.- Looking at backtrace of panic task, it shows panic was occurred in module "sisips".- The kernel panic occurred in sisips module provided by Symantec while dereferencing a null value in a register.- Red Hat does not ship this module and hence it is not possible to analyse the code for this module.- Engage module vendor of sisips and take their opinion on this issue. +++++Kernel crashed at _ZN7Process13listToProcessEP15LIST_ENTRY_LINK+0x0/0x10 [sisips]https://access.redhat.com/solutions/3070771+++++  ~~~~         KERNEL: /cores/retrace/repos/kernel/x86_64/usr/lib/debug/lib/modules/2.6.18-434.el5/vmlinux  [TAINTED]    DUMPFILE: /cores/retrace/tasks/744071135/crash/vmcore  [PARTIAL DUMP]        CPUS: 16        DATE: Wed Feb 16 13:42:10 EST 2022      UPTIME: 01:57:56LOAD AVERAGE: 1.14, 1.31, 1.38       TASKS: 894    NODENAME: dc3cnz     RELEASE: 2.6.18-434.el5     VERSION: #1 SMP Thu Aug 23 15:28:29 EDT 2018     MACHINE: x86_64  (3465 Mhz)      MEMORY: 189.4 GB       PANIC: "Unable to handle kernel NULL pointer dereference at 0000000000000000 RIP: "         PID: 18875     COMMAND: "tnslsnr"        TASK: ffff81232d4220c0  [THREAD_INFO: ffff81231bd60000]         CPU: 15       STATE: TASK_RUNNING (PANIC)  crash> mod -t | grep Uiomemory_vsl             41(U)sisips                   41(U)sisfim                   41(U)crash> crash> crash> btPID: 18875  TASK: ffff81232d4220c0  CPU: 15  COMMAND: "tnslsnr" #0 [ffff81231bd61a30] crash_kexec at ffffffff800bafc8 #1 [ffff81231bd61af0] __die at ffffffff80066197 #2 [ffff81231bd61b30] do_page_fault at ffffffff80068750 #3 [ffff81231bd61c20] error_exit at ffffffff800659f1    [exception RIP: strcpy+8]    RIP: ffffffff801651c2  RSP: ffff81231bd61cd8  RFLAGS: 00010286    RAX: ffff81233980322f  RBX: ffff81301c45d4c0  RCX: ffff81233980326f    RDX: 0000000000000000  RSI: ffff812339803241  RDI: 0000000000000000    RBP: 0000000000000031   R8: 726e736c736e742f   R9: 0000000000000001    R10: ffff81231bd61c98  R11: ffff81231bd61ca0  R12: ffff812339803240    R13: ffff81231f0f9000  R14: ffff81231bd61e20  R15: 00000000000049bb    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018 #4 [ffff81231bd61cd8] wcscpy at ffffffff8862d8f9 [sisips] #5 [ffff81231bd61ce8] _ZN12SimpleStringaSEPKc at ffffffff8864bc46 [sisips] #6 [ffff81231bd61d08] _ZN7Process14setCommandLineER19UnicodeStringBuffer at ffffffff88651989 [sisips] #7 [ffff81231bd61d28] _ZN7Process14setCommandLineEPKc at ffffffff886519e4 [sisips] #8 [ffff81231bd61d58] _Z21SetProcessCommandLineP7Process at ffffffff88635d0e [sisips] #9 [ffff81231bd61d88] _Z12SetUpProcessP7Process at ffffffff88636f22 [sisips]#10 [ffff81231bd61dc8] _ZN13ProcessCommon10GetProcessEi at ffffffff88656419 [sisips]#11 [ffff81231bd61de8] AppfireCreateProcess at ffffffff88637a0f [sisips]#12 [ffff81231bd61e98] hook_execve at ffffffff8862c5bf [sisips]#13 [ffff81231bd61f80] tracesys at ffffffff80069e8a (via system_call)    RIP: 00000033f889a557  RSP: 00007fffe10181d8  RFLAGS: 00000202    RAX: ffffffffffffffda  RBX: ffffffff80069e8a  RCX: ffffffffffffffff    RDX: 0000000012cbaf00  RSI: 0000000012c71380  RDI: 0000000012c89cd0    RBP: 00007fffe1018240   R8: fefefefefefefeff   R9: 2f2f2f2f2f2f2f2f    R10: 0000000000000000  R11: 0000000000000202  R12: ffff81232d4220c0    R13: ffff81232d4220c0  R14: ffff81231f0f9000  R15: ffff81231f0f9000    ORIG_RAX: 000000000000003b  CS: 0033  SS: 002bcrash>    SISFIM: EXT3 file hooks enabledUnable to handle kernel NULL pointer dereference at 0000000000000000 RIP:  [<ffffffff801651c2>] strcpy+0x8/0x15PGD 80000023255ed067 PUD 23237eb067 PMD 0 Oops: 0002 [1] SMP last sysfs file: /devices/pci0000:00/0000:00:03.0/0000:11:00.0/host1/rport-1:0-0/target1:0:0/1:0:0:15/timeoutCPU 15 Modules linked in: sisfim(PU) hidp l2cap bluetooth bonding ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink iptable_filter ip_tables ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio cxgb3i libcxgbi cxgb3 8021q libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi ipv6 xfrm_nalgo crypto_api sisips(PU) dm_round_robin dm_multipath scsi_dh raid1 video backlight sbs power_meter hwmon i2c_ec i2c_core dell_wmi wmi button battery asus_acpi acpi_memhotplug ac parport_pc lp parport sg joydev iomemory_vsl(PU) i7core_edac edac_mc hpilo serio_raw tpm_tis tpm bnx2 tpm_bios e1000e pcspkr dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod ahci libata usb_storage qla2xxx scsi_transport_fc shpchp cciss sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcdPid: 18875, comm: tnslsnr Tainted: P     -------------------- .....

Cause

The crash was caused (as per the stack trace) because of a command line string pointer being NULL or having no value when passed back to the module handling the request.

Environment

Agent Versions: 5.2.9x, 6.9.1

 

Resolution

The agent was a much older version but maintained for various customers.  The fixed version is 5.2.9.MP6.HF6_5.2.9.982 which is available through Broadcom Data Server Protection (DCS) Technical Support.