How to increase maxIntrCookies if default limit is reached in ESXi ?

Products

• VMware vSphere ESXi

Issue/Introduction

This article will provide a workaround to increase the number of interrupt cookies.

Symptoms:

1. Powering on a virtual machine with PCI Passthrough devices fails:                                                                              

•   In the vmware.log file of the affected virtual machine, you see entries similar to
    ####:##:##.# failed to register interrupt                                      

•   In the vmkernel.log file of the ESXi host where the virtual machine is running, you see
     entries similar to: 
     Unable to find a free interrupt number                    

2. Detection of devices fails while booting ESXi.

•   In the vmkernel.log file of the ESXi host, you see entries similar to: 

    Unable to find a free interrupt number

3. ESXi doesn't detect all the vmnic adapters, But, those vmnic adapters are seen in "lspci", not on esxcfg-nics -l

•   In the vmkernel.log file of the ESXi host, you see entries similar to: 

    ####:##:##.# failed to allocate 64 MSIX interrupts

Environment

• VMware vSphere ESXi 7.0.x
• VMware vSphere ESXi 8.0.x
• VMware vSphere ESXi 9.0.x
• VMware vSphere ESX 9.1

Cause

• This issue occurs due to an ESXi limit on interrupt resources. When PCI Passthrough devices request many interrupts to be allocated, it is possible that this limit has been reached which causes the issue.

Resolution

• To address this issue update host "maxIntrCookies" settings to 4096 or 32768 depending on version.
• The "maxIntrCookies" default value was increased to 4096 in ESX version 8.0 U3. The "maxIntrCookies" max configurable value was increased to 32768 in ESX 9.1
• The default value for "maxIntrCookies" depends on the ESXi version in use and whether the advanced settings value was manually set higher before a host was upgraded to ESXi version 8.0U3 where the default value was increased.  

For more information on instructions on how to configure boot options, see Configuring advanced options for ESXi/ESX (310338).