Driver for RAID controller PERC H965i Front cannot start due failed to allocate interrupts. status=Out of resources.
search cancel

Driver for RAID controller PERC H965i Front cannot start due failed to allocate interrupts. status=Out of resources.

book

Article ID: 370423

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

The article will provide a workaround if RAID controller have not enough interrupts.

 

Symptoms:

Fresh installation of ESXi with PERC H965i Front only possible by applying boot option maxIntrCookies=4096.

Once installed, no local datastore is available in vSphere.

Rebooting the host ends up in resetting the configuration of the management network and other configuration made to default.

Adjustment of the boot option via esxcli system settings kernel set -s=maxIntrCookies -v=4096 can be confirmed by esxcli system settings kernel list | grep maxInt.

bootbank located under /tmp

 

vmkenel log
YYYY-MM-DDTHH:MM:SS.233Z In(182) vmkernel: cpu0:2097152)PCIE: 694: 0000:5b:00.0: PCIe v2 PCI Express Endpoint
YYYY-MM-DDTHH:MM:SS.233Z In(182) vmkernel: cpu0:2097152)PCI: 387: Found onboard instance 0x8101 for 0000:5b:00.0
YYYY-MM-DDTHH:MM:SS.233Z In(182) vmkernel: cpu0:2097152)PCI: 404: Found description "SL1 RAID" from SMBIOS entry of 0000:5b:00.0 for 0000:5b:00.0
YYYY-MM-DDTHH:MM:SS.233Z In(182) vmkernel: cpu0:2097152)PCI: 1045: 0000:5b:00.0: probing 1000:00a5 1028:2115 0104 0001
...
YYYY-MM-DDTHH:MM:SS.234Z In(182) vmkernel: cpu0:2097152)PCI: 1299: 0000:5b:00.0: registering 1000:00a5 1028:2115
...
YYYY-MM-DDTHH:MM:SS.030Z In(182) vmkernel: cpu15:2098125)PCI: 1068: 0000:5b:00.0 named 'vmhba0' (was '')
...
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3510: Instance address: 0x431a1e6012d0
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=0, flags=0xc, start=0xbf000000, size=0x4000
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=1, flags=0x0, start=0x0, size=0x0
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=2, flags=0x0, start=0x0, size=0x0
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=3, flags=0x0, start=0x0, size=0x0
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=4, flags=0x0, start=0x0, size=0x0
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=5, flags=0x0, start=0x0, size=0x0
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)VMK_PCI: 770: 0000:5b:00.0: pciBar 0 bus_addr 0xbf000000 size 0x4000
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: disable_ioc_state: 398: Enable IOC=0x1, Ready=0x1, timeout=510
...
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_populate_iocfacts: 1288: IOC ThrottleInfo -> DataLength=17, HighBandwidth=128, LowBandwidth=64
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_populate_iocfacts: 1302: maxDataLen=1024KB, maxSGEs=256
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_update_mpool: 4224: Mem Pool size changed from 100MB to 150MB
YYYY-MM-DDTHH:MM:SS.476Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_allocate_request_pool: 1974: Max Outstanding Requests=8192, Max Targets=1025
YYYY-MM-DDTHH:MM:SS.491Z In(182) vmkernel: cpu120:2098126)bcm_mpi3_0000:5b:00.0: mpi3_setup_interrupts: 2821: Current msix vectors = 128, CPUs in the system = 128 
YYYY-MM-DDTHH:MM:SS.493Z Wa(180) vmkwarning: cpu120:2098126)WARNING: IntrCookie: 1458: Unable to find a free interrupt number; see https://knowledge.broadcom.com/external/article?legacyId=78182  
YYYY-MM-DDTHH:MM:SS.494Z Wa(180) vmkwarning: cpu120:2098126)WARNING: VMK_PCI: 593: 0000:5b:00.0: failed to allocate 128 MSIX interrupts
YYYY-MM-DDTHH:MM:SS.494Z Wa(180) vmkwarning: cpu120:2098126)WARNING: bcm_mpi3_0000:5b:00.0: mpi3_setup_interrupts: 2832: failed to allocate interrupts. status=Out of resources.
YYYY-MM-DDTHH:MM:SS.494Z Al(177) vmkalert: cpu120:2098126)ALERT: bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3691: failed to setup interrupts 

 

Environment

VMware vSphere ESXi 7.0.0
VMware vSphere ESXi 8.0.0

Cause

This issue occurs due to an ESXi limit on interrupt resources.

Resolution

VMware is aware of this issue and working to resolve this in a future release.

 

Workaround:

1. Install the system and hit SHIFT+O during the procedure to set boot parameter “maxIntrCookies=4096”

2. Once the system performs the first reboot hit SHIFT+O again and set the parameter “maxIntrCookies=4096”

3. When the boot process has been completed go to ESX Cli and use “vi /bootbank/boot.cfg” to edit the boot.cfg

4. Put the option “maxIntrCookies=4096” at the end of the kernelopt section

5. Repeat the steps 3 & 4 for the /altbootbank.cfg 

6. Reboot the system

7. Check via “ls -l” if the bootbank is now located on the local datastore

 

 

Once the actions are completed please configure the management network and see if it is also persistent after a reboot. On the system in Lab all steps have been completed and the system is working fine now. The only drawback here is, that once the systems are upgraded to a higher version of ESXi these settings might get lost as a new bootbank is created. Then steps need to be repeated.

 

Powered by Wolken Software