PSOD: Zerto driver Module Triggers Memory Corruption Error on ESXi

Products

• VMware vSphere ESXi

Issue/Introduction

• Purple Screen of Death (PSOD) occurred on the ESXi Host, presenting the following diagnostic trace

     Backtrace for 8.x 

        0x45399e41b500 : [0x42002ff7c280]PanicvPanic Int@vmkernel#nover+0x20c stack: 0x4309fe490868

    0x45399e41b5b0 : [0x42002ff7c9fc]Panic_NoSave@vmkernel#nover+0x4d stack: 0x45399e41b610

    0x45399e41b610: [0x42002ff32db3]DLM_free@vmkernel#nover+0x15c stack: 0x431b3ee02fd0

    0x45399e41b630 : [0x42002ff3b8ad]Heap_Free@vmkernel#nover+0xb2 stack: 0x431b3ee02fd8

    0x45399e41b680 : [0x420032681df7]_zcmd_free_cmd@(zdriver_ESX_80U2_2b1a507d)#<None>+0x18 stack: 0x453900000002

    base fs=0x0 gs=0x420041800000 Kgs=0x0

 

       OR

       YYYY-MM-DDT01:34:48.099Z cpu30:2201855)Backtrace for current CPU #30, worldID=2201855, fp=0x45ca800e4640
       YYYY-MM-DDT01:34:48.099Z cpu30:2201855)0x453a74892fc0:[0x42002e6b1aa0]zfl_cmd@(zdriver_ESX_80U2_6f257fcf)#<None>+0x58 stack: 0x0, 0x0, 0x0, 0x0, 0x0
       YYYY-MM-DDT01:34:48.123Z cpu30:2201855)ESC[45mESC[33;1mVMware ESXi 8.0.3 [Releasebuild-24585383 x86_64]ESC[0m
       #DF Exception 8 IP 0x42002e6b1aa0 SP 0x453a74892fc0

       Module(s) involved in panic: [zdriver_ESX_80U2_6f257fcf  (External)]
       YYYY-MM-DDT01:34:48.125Z cpu30:2201855)@BlueScreen: #DF Exception 8 IP 0x42002e6b1aa0 SP 0x453a74892fc0
       YYYY-MM-DDT01:34:48.126Z cpu30:2201855)Code start: 0x42002c400000 VMK uptime: 133:11:53:34.738
       YYYY-MM-DDT01:34:48.126Z cpu30:2201855)0x453a74892fc0:[0x42002e6b1aa0]zfl_cmd@(zdriver_ESX_80U2_6f257fcf)#<None>+0x58 stack: 0x0
       YYYY-MM-DDT01:34:48.126Z cpu30:2201855)0xfffffffffc407d68:[0xfffffffffc067f5a] (vmm64)
       YYYY-MM-DDT01:34:48.126Z cpu30:2201855)0xfffffffffc407df0:[0xfffffffffc068335]

        Backtrace for 7.x

 

        0x4539bde9b7c8:[0x42001551a62b]_vmk_Memcpy@vmkernel#nover+0x27 stack: 0x4200157afdc6, 0x450000000001, 0x43026f203228, 0xffffffffb441f301, 0x43204be03368

    0x4539bde9b7d0:[0x42001743a2a3]zcmd_os_buf_to_zcmd@(zdriver_ESX_70U3_de963296)#<None>+0x6c stack: 0x450000000001, 0x43026f203228, 0xffffffffb441f301, 0x43204be03368, 0x1

Environment

• ESXi 7.x
• ESXi 8.x

Cause

According to the memory dump, the PSOD resulted from a corruption within zstheap0. An inconsistency was detected at particular memory address when a function within the zdriver module attempted to deallocate a memory chunk, thereby initiating the PSOD

Resolution

This is not a Broadcom issue. For further assistance engage HPE Zerto Support