PSOD on ESXi with FC_FlowPktAttrSet

Products

• VMware NSX

Issue/Introduction

• The ESXi host experienced a crash with below backtrace:
  

  Sample of PSOD backtrace

  ##############:[##############]Pkt_AttrObjAlloc@vmkernel#nover+#### stack: #### ##############:[##############]Pkt_AttrCreateAndSet@vmkernel#nover+#### stack: ############## ##############:[##############][e-mail]############### stack: ############## ##############:[##############][e-mail]############### stack: ############## ##############:[##############][email protected]############### stack: #### ##############:[##############]IOChain_Resume@vmkernel#nover+############## stack: ############## ##############:[##############]Port_InputResume@vmkernel#nover+#### stack: #### ##############:[##############]Vmxnet3VMKDevTQDoTx@vmkernel#nover+#### stack: ############## ##############:[##############]Vmxnet3VMKDev_AsyncTx@vmkernel#nover+#### stack: #### ##############:[##############]NetWorldPerVMCB@vmkernel#nover+#### stack: ############## ##############:[##############]CpuSched_StartWorld@vmkernel#nover+#### stack: #### ##############:[##############]Debug_IsInitialized@vmkernel#nover+#### stack: ####

• The affected ESXi host is prepared for NSX.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX

Cause

A memory corruption in the ESXi PCPU triggered the PSOD.

Resolution

This is a known issue impacting VMware NSX.

 

Workarounds:

• The remediation workaround (after PSOD) is to reboot the ESXi host.
• The preventative workaround (to avoid a PSOD) is to disable NSX Flow-Cache.
  Note: Disabling NSX Flow-Cache will have an impact on performance.
  
  To disable NSX Flow-Cache from ESXi CLI, use one of the following methods:
  • Option A:
    

    Disable NSX Flow-Cache: nsxdp-cli fc disable Make the change persistent: net-dvs --persist

  • Or option B:
    

    Backup the current configuration: cp /etc/vmware/nsx/nsx-cfgagent.xml /etc/vmware/nsx/nsx-cfgagent.xml.bak Edit the Flow-Cache configuration: vi /etc/vmware/nsx/nsx-cfgagent.xml Change both flags to false, as below: <flowCache> <enabled>false</enabled> <mcastEnabled>false</mcastEnabled> </flowCache> Restart the nsx-cfgagent service: /etc/init.d/nsx-cfgagent restart