ESXi 7.0.x reporting out of memory errors for vmware-usbarbitrator in vmkernel.log on on HPE servers
search cancel

ESXi 7.0.x reporting out of memory errors for vmware-usbarbitrator in vmkernel.log on on HPE servers

book

Article ID: 326255

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:
  • The issue is primarily seen on HPE Servers
  • In some cases, the affected ESXi host appears as not responding in vCenter Inventory
  • VMs could restart after being failed over by vSphere HA
  • The ESXi host may appear normal, you can connect via SSH, run esxcli commands etc
  • vmkernel logs are contain a high number of "Cannot allocate memory" messages like the below:
2022-04-25T05:42:47.916Z cpu55:2099016)WARNING: UserObj: 4914: vmware-usbarbitrator: Poll on fd 10 failed: Out of memory
2022-04-25T05:24:07.390Z error hostd[2100397] [Originator@6876 sub=Libs] BmcInfo: poll(43), failed 12 Cannot allocate memory
2022-04-25T05:44:09.226Z error hostd[2100572] [Originator@6876 sub=Default] Node 0: poll(fd=32), errno=12 Cannot allocate memory


Environment

VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 7.0.3

Cause

This is not a VMware issue.
The memory allocation issue is being caused by the HPE SUT (Integrated Smart Update Tool) utility.

Resolution

VMware and HPE Engineering teams are working together to find the exact root cause and provide a permanent solution in a future release of the HPE SUT utility.

For more details on this issue and prevent it from impacting your environment, please engage the HPE support.

 

 


Workaround:

As a workaround to prevent this memory allocation issue, you can uninstall the HPE SUT component.

  • De-Register the SUT utility:
    # sut -deregister
  • Uninstall the SUT component:
    # esxcli software component remove –n sutComponent

 


Additional Information

To verify if the SUT component is installed on the ESXi, use the following command:
# esxcli software component list | grep -i sut
The output should will show:
sutComponent       Integrated Smart Update Tool