"SwSec_DestroyFilter" backtrace in PSOD for ESXi Host prepared for NSX
search cancel

"SwSec_DestroyFilter" backtrace in PSOD for ESXi Host prepared for NSX

book

Article ID: 392851

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • ESXi in NSX cluster crashes with PSOD backtrace as below illustrated below.
Panic from another CPU (cpu 119, world 2098928): ip=######### randomOff=0xe00000:Spin count exceeded - possible deadlock with PCPU 73Halting PCPU 119.2025-03-19T10:05:04.117Z cpu73:6977316)ESC[45mESC[33;1mVMware ESXi 7.0.3 [Releasebuild-24585291 x86_64]ESC[0m
NMI IPI: Panic requested by another PCPU. RIPOFF(base):RBP:CS [0x2c12b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:05:04.147Z cpu73:6977316)cr0=0x80010031 cr2=0x6fca929d92 cr3=0x29778b6000 cr4=0x142768
2025-03-19T10:05:04.152Z cpu73:6977316)FMS=06/6a/6 uCode=0xd0003b9
*PCPU73:6977316/vmx
PCPU  0: SSVVVUUVSSUVVVVVUVSVVVSVUVVVUSVVVVVSSVVVVVSVVVVSUUVVVVVVSVUVVVSS
PCPU 64: UVUSVUVVUUVUVVSVVUVSVVSVVVVVUVSVVVUVVVVVVVSUUVRVVVVVVVSVSVVVSVVSU
2025-03-19T10:05:04.187Z cpu73:6977316)Code start: 0x420000e00000 VMK uptime: 9:17:33:24.513
2025-03-19T10:05:04.196Z cpu73:6977316)Saved backtrace from: pcpu 73 SpinLock spin out NMI
2025-03-19T10:05:04.210Z cpu73:6977316)0x453a14a9bb70:[0x420000e2c12a]VmkTimerSpin@vmkernel#nover+0x2b stack: 0x26784317dbc1d800
2025-03-19T10:05:04.225Z cpu73:6977316)0x453a14a9bbb0:[0x420000e2d015]vmk_TimerCancel@vmkernel#nover+0x1a6 stack: 0x10000000600024f
2025-03-19T10:05:04.245Z cpu73:6977316)0x453a14a9bc00:[0x4200025b3110][email protected].switchsecurity#1.0.7.0.24476730+0x129 stack: 0x431eda4e2430
2025-03-19T10:05:04.264Z cpu73:6977316)0x453a14a9bc60:[0x42000242c26d][email protected]#1.0.7.0.24476730+0x76 stack: 0x43069c9631d8
2025-03-19T10:05:04.277Z cpu73:6977316)0x453a14a9bcb0:[0x42000104443f]NetEvent_PostEvent@vmkernel#nover+0x198 stack: 0x0
2025-03-19T10:05:04.293Z cpu73:6977316)0x453a14a9bd40:[0x42000106ed78]Port_PostEventWithCtx@vmkernel#nover+0x109 stack: 0x4306b9f44f40
2025-03-19T10:05:04.307Z cpu73:6977316)0x453a14a9bd90:[0x4200010780f2]Portset_DisconnectPort@vmkernel#nover+0x26b stack: 0x3fff
2025-03-19T10:05:04.321Z cpu73:6977316)0x453a14a9bdf0:[0x4200010bea6e]NetDisconnect@vmkernel#nover+0x22b stack: 0x6f88c772b0
2025-03-19T10:05:04.335Z cpu73:6977316)0x453a14a9be60:[0x4200010c0f9c]Net_Disconnect@vmkernel#nover+0x1a9 stack: 0x42000132ec28
2025-03-19T10:05:04.351Z cpu73:6977316)0x453a14a9bec0:[0x420001319ace]UWVMKSyscall_NetDisconnect@vmkernel#nover+0x37 stack: 0x453a14a9bf40
2025-03-19T10:05:04.366Z cpu73:6977316)0x453a14a9bee0:[0x4200012b542e]User_UWVMK64SyscallHandler@vmkernel#nover+0x183 stack: 0x0
2025-03-19T10:05:04.378Z cpu73:6977316)0x453a14a9bf40:[0x420000f4b638]SyscallUWVMK64@vmkernel#nover+0x90 stack: 0x0
2025-03-19T10:05:04.389Z cpu73:6977316)base fs=0x0 gs=0x420052400000 Kgs=0x0
2025-03-19T10:05:04.394Z cpu73:6977316)1 other PCPU is in panic.
2025-03-19T10:04:44.649Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x131ba2(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.628Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c12b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.622Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c17b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.580Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c180(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.563Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c17b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.544Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c12b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.500Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c12b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:45.060Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c12b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.963Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x3045c(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)
2025-03-19T10:04:44.680Z cpu73:6977316)NMI: 712: NMI IPI: RIPOFF(base):RBP:CS [0x2c17b(0x420000e00000):0x4317dbc0fbc0:0xf48] (Src 0x4, CPU73)

 

Environment

VMware NSX

Cause

  • The issue is caused due to a race condition between the thread handling the deletion or cleanup of the switch security "enable" property, and a timer thread callback that was in the process of being cancelled.
  • This behavior can occur during scenarios such as, but not limited to:
    • VNIC deletion or reconnection.
    • Virtual machine power-off.
    • Switch security (SwSec) property cleanup on the source host during vMotion. 

Resolution

This issue is resolved in VMware NSX 4.2.2 and 4.2.1.4, available at Broadcom downloads.
If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

Workaround:

  • There is no workaround to prevent recurrence of this issue.

Additional Information

You can match the issue to the KB article by:

  • Check the /var/run/log/logEFI.log in the ESXi logs.
  • Check the /var/run/log/vmkernel.log in the ESXi logs.
  • Match The backtrace from the PSOD screen.
Powered by Wolken Software