VM using Directpath I/O or SR-IOV experiences a soft lock up and goes unresponsive

Products

• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 8.0

Issue/Introduction

• VM using Directpath I/O or SR-IOV experiences a soft lock up and goes unresponsive
• This may recover on its own or would require a reset.

Cause

A rare race condition between the interrupt virtualization and the VMkernel CPU scheduler in  VMs with Directpath I/O device might result in guest kernel soft lockups.

 

Resolution

Resolved in VMware ESXi 8.0 Update 3, available under Download Broadcom products and software

Workarounds

• ESXi 7.0.x Workaround
• ESXi 8.0.x Workaround

 

ESXi 7.0.x Workaround

1. Open a SSH session to ESXi  host.

2. Check the current value of the option vtdEnableIntrVirt in the ESXi boot-time options in the VMkernel.boot.* namespace,using the following command:

   # esxcfg-advcfg --get-kernel vtdEnableIntrVirt

    

3. Set a new value for an option using the esxcfg-advcfg command:

   # esxcfg-advcfg --set-kernel "FALSE" vtdEnableIntrVirt

4. Verify that the option was correctly set:

   # esxcfg-advcfg --get-kernel vtdEnableIntrVirt

5. Reboot the ESXi host to apply the changed setting.

6. Once the server is rebooted, run the command  esxcfg-advcfg --get-kernel vtdEnableIntrVirt again, to verify that the value still shows as FALSE.

 

ESXi 8.0.x Workaround

1. Open a SSH session to ESXi  host. 
2. Check the current value of the option iovEnablePostedIntr in the ESXi boot-time options in the VMkernel.boot.* namespace, using the following command:

   # esxcfg-advcfg --get-kernel iovEnablePostedIntr

3. Set a new value for an option using the esxcfg-advcfg command:

   # esxcfg-advcfg --set-kernel "FALSE" iovEnablePostedIntr

4. Verify that the option was correctly set:

   # esxcfg-advcfg --get-kernel iovEnablePostedIntr

5. Reboot the ESXi host to apply the changed setting.
6. Once the server is rebooted, run the command esxcfg-advcfg --get-kernel iovEnablePostedIntr to verify if the value still shows as FALSE


Additional Information

Note: In VMware vSphere ESXi 7.0 Update 2 and later 7.0 version, the default value of vtdEnableIntrVirt is set to TRUE.