SFCB crashing and generating sfcb-vmware_bas-zdump core dumps due to large number of PCI devices on host

Products

• VMware vSphere ESXi

Issue/Introduction

To prevent sfcb-vmware_base provider from crashing.

Symptoms:

SFCB crashing and generating core dumps similar to: sfcb-vmware_bas-zdump.001

VMkernel logs report following errors:

MemSchedAdmit: 471: Admission failure in path: vmware_base/sfcb-ProviderMa.2104983/uw.2104983
MemSchedAdmit: 478: UserWorld 'sfcb-vmware_bas' with cmdline '/bin/sfcbd'
MemSchedAdmit: 489: uw.2104983 (17337) extraMin/extraFromParent: 782/782, vmware_base (15122) childEmin/eMinLimit: >20204/20480

 

Syslog.log reports following errors:

sfcb-vmware_base[2104983]: tool_mm_realloc_or_die: memory re-allocation failed(orig=1600000 new=3200000 msg=Cannot >allocate memory, aborting
sfcb-ProviderManager[2104509]: handleSigChld:166665024 provider terminated, pid=2104983, exit=0 signal=6

Note: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 7.0.2
VMware vSphere ESXi 7.0.3
VMware vSphere ESXi 7.0.1
VMware vSphere ESXi 7.0.0

Cause

SFCB vmware_base provider needs a bigger resource pool while enumerating large number of PCI devices.

Resolution

The issue has been resolved by increasing the default pool size for sfcb-vmware_base provider from ESXi 7.0 Update 3i build 20842708 or later.

Workaround:
The resource pool for vmware_base can be manually adjusted in one of the following ways:

esxcli system wbem set --rp-override="vmware_base=120"

OR

By Editing the config file:
echo "provMemOveride:vmware_base=120" >> /etc/sfcb/sfcb.cfg && esxcli system wbem set -e 0 && esxcli system wbem set -e 1

Additional Information

Impact/Risks:

The SFCB query to enumerate PCI devices fails and causes SFCB vmware_base zdump.