DLM_Free / Heap_Free PSOD in VLANMTUCheck Module

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Host experiences PSOD, below is the snippet of the backtrace is reported:

[0x42003714512f]NMVCFireUnicastTicket@(vlanmtucheck)#<None>+0x48 stack: 0x430481c02f88....
......

[0x420037147827]NMVC_UplinkEventCB@(vlanmtucheck)#<None>+0x47c stack: 0x1004305be6104c0
[0x42003621572d]NetEvent_PostEvent@vmkernel#nover+0x172 ... [0x42003624c12b]Portset_BroadcastEvent@vmkernel#nover ...
cpuXXX:2098814)0x453ba891be70:[0x420036262b78]UplinkTreeNotify@vmkernel#nover+0x91 stack: 0x4312f0c01d2c
.....

[0x4200362686a6]UplinkDev_UpdateLinkState@vmkernel#nover+0x8f stack: 0xb1d
cpuXXX:2098814)0x453ba891bf20:[0x420036cd7a7c]ixgben_ReportLinkStatus@(ixgben)#
....

cpuXXX:2098814)0x453ba891bf40:[0x420036cdb3a2]ixgben_CheckAsyncEvents@(ixgben)#

• Leading to the PSOD, we see below errors in vmkernel.log:

YYYY-MM-DDThh:mm:ss.557Z cpuXXX:2097579)VLANMTUCheck: NMVCFireUnicastTicket:819: add ticket failed in state 1:Limit exceeded
YYYY-MM-DDThh:mm:ss.557Z cpuXXX:2097579)VLANMTUCheck: NMVC_StartPsReq:2551: failed to start uplink request for uplink vmnicX:Limit exceeded
YYYY-MM-DDThh:mm:ss.557Z cpuXXX:2097579)VLANMTUCheck: NMVCFireUnicastTicket:819: add ticket failed in state 1:Limit exceeded
YYYY-MM-DDThh:mm:ss.557Z cpuXXX:2097579)VLANMTUCheck: NMVC_StartPsReq:2551: failed to start uplink request for uplink vmnicX:Limit exceeded
YYYY-MM-DDThh:mm:ss.134Z cpuXXX:2098814)VLANMTUCheck: NMVCAddUplink:1975: Add uplink request for uplink vmnicX:DvsPortset-X
YYYY-MM-DDThh:mm:ss.134Z cpuXXX:2098814)VLANMTUCheck: NMVCMTULocalChkEventPost:230: local MTU setting MATCHED on uplink vmnicX(DV port X).

Environment

VMware vSphere ESXi 7.0.0

Cause

• It is a heap double free issue related to Health Check component.
• Issue is generally (considered to be a rare corner case) seen when a new uplink is added to vDS while the Health Check features (vLAN-MTU Health Check / Teaming and Failover Health Check / Beacon Probing) are enabled.

Resolution

Issue has been fixed in 7.0 U3.

Workaround:
Before adding any new uplink to the vDS, disable the Health Check Features (vLAN-MTU Health Check / Teaming and Failover Health Check / Beacon Probing).