Driver for PERC H965i Front RAID Controller Fails to Start with error 'Out of Resources' 'Failed to Allocate Interrupts'
search cancel

Driver for PERC H965i Front RAID Controller Fails to Start with error 'Out of Resources' 'Failed to Allocate Interrupts'

book

Article ID: 370423

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • This article outlines a solution for handling RAID controller interrupt shortages
  • Fresh installation of ESXi with PERC H965i Front only possible by applying boot option maxIntrCookies=4096.
  • Once installed, no local datastore is available in vSphere.
  • Rebooting the host causes the management network and other settings to revert to their default configurations.
  • Adjustment performed via esxcli system settings kernel set -s=maxIntrCookies -v=4096 can be confirmed by running the command esxcli system settings kernel list | grep maxInt.
  • bootbank located under /tmp




  • /var/run/log/vmkenel log in ESXi reports the below error stack: 
    In(182) vmkernel: cpu0:2###2)PCIE: 694: 0000:5b:00.0: PCIe v2 PCI Express Endpoint
In(182) vmkernel: cpu0:2###2)PCI: 387: Found onboard instance 0x8101 for 0000:5b:00.0
In(182) vmkernel: cpu0:2###2)PCI: 404: Found description "SL1 RAID" from SMBIOS entry of 0000:5b:00.0 for 0000:5b:00.0
In(182) vmkernel: cpu0:2###2)PCI: 1045: 0000:5b:00.0: probing 1000:00a5 1028:2115 0104 0001

In(182) vmkernel: cpu0:2###2)PCI: 1299: 0000:5b:00.0: registering 1000:00a5 1028:2115

In(182) vmkernel: cpu15:2###5)PCI: 1068: 0000:5b:00.0 named 'vmhba0' (was '')

In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3510: Instance address: 0x431a1e6012d0
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=0, flags=0xc, start=0xbf000000, size=0x4000
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=1, flags=0x0, start=0x0, size=0x0
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=2, flags=0x0, start=0x0, size=0x0
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=3, flags=0x0, start=0x0, size=0x0
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=4, flags=0x0, start=0x0, size=0x0
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3566: bar=5, flags=0x0, start=0x0, size=0x0
In(182) vmkernel: cpu120:2####6)VMK_PCI: 770: 0000:5b:00.0: pciBar 0 bus_addr 0xbf000000 size 0x4000
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: disable_ioc_state: 398: Enable IOC=0x1, Ready=0x1, timeout=510

In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_populate_iocfacts: 1288: IOC ThrottleInfo -> DataLength=17, HighBandwidth=128, LowBandwidth=64
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_populate_iocfacts: 1302: maxDataLen=1024KB, maxSGEs=256
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_update_mpool: 4224: Mem Pool size changed from 100MB to 150MB
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_allocate_request_pool: 1974: Max Outstanding Requests=8192, Max Targets=1025
In(182) vmkernel: cpu120:2####6)bcm_mpi3_0000:5b:00.0: mpi3_setup_interrupts: 2821: Current msix vectors = 128, CPUs in the system = 128 
Wa(180) vmkwarning: cpu120:2####6)WARNING: IntrCookie: 1458: Unable to find a free interrupt number; see https://knowledge.broadcom.com/external/article?legacyId=78182  

Wa(180) vmkwarning: cpu120:2####6)WARNING: VMK_PCI: 593: 0000:5b:00.0: failed to allocate 128 MSIX interrupts
Wa(180) vmkwarning: cpu120:2####6)WARNING: bcm_mpi3_0000:5b:00.0: mpi3_setup_interrupts: 2832: failed to allocate interrupts. status=Out of resources.
Al(177) vmkalert: cpu120:2####6)ALERT: bcm_mpi3_0000:5b:00.0: mpi3_drv_attach_device: 3691: failed to setup interrupts

Environment

VMware vSphere ESXi 7.0.0
VMware vSphere ESXi 8.0.0

Cause

This issue occurs due to an ESXi limit on interrupt resources.

Resolution

VMware is aware of this issue and working to resolve this in a future release.

Workaround:

  1. Install the system and hit SHIFT+O during the procedure to set boot parameter “maxIntrCookies=4096”
  2. Once the system performs the first reboot hit SHIFT+O again and set the parameter “maxIntrCookies=4096”
  3. When the boot process has been completed go to ESX Cli and use “vi /bootbank/boot.cfg” to edit the boot.cfg
  4. Put the option “maxIntrCookies=4096” at the end of the kernelopt section


  5. Repeat the steps 3 & 4 for the /altbootbank.cfg
  6. Reboot the system
  7. Check via “ls -l” if the bootbank is now located on the local datastore



  8. Once the actions are completed please configure the management network and see if it is also persistent after a reboot.

Additional Information

On the system in Lab all steps have been completed and the system is working fine now. The only drawback here is, that once the systems are upgraded to a higher version of ESXi these settings might get lost as a new bootbank is created. Then steps need to be repeated.

 

Powered by Wolken Software