SFCB crashing and generating dumps: sfcb-vmware_bas-zdump

Products

• VMware vSphere ESXi

Issue/Introduction

To prevent SFCB from crashing.

Symptoms:

• On VMware vSphere ESXi 6.7 hosts we see SFCB crashing and generating dumps similar to: sfcb-vmware_bas-zdump.001
• VMkernel logs report following errors:

2020-02-08T03:51:41.503Z cpu19:2268488)MemSchedAdmit: 471: Admission failure in path: vmware_base/sfcb-ProviderMa.2268485/uw.2268485
2020-02-08T03:51:41.503Z cpu19:2268488)MemSchedAdmit: 478: UserWorld 'sfcb-vmware_bas' with cmdline '/bin/sfcbd'
2020-02-08T03:51:41.503Z cpu19:2268488)MemSchedAdmit: 489: uw.2268485 (782825) extraMin/extraFromParent: 196/196, vmware_base (782805) childEmin/eMinLimit: 38331/38400

• Syslog.log reports following :

2020-02-08T04:13:19Z sfcb-vmware_base[2268485]: tool_mm_realloc_or_die: memory re-allocation failed(orig=200000 new=400000 msg=Cannot allocate memory, aborting
2020-02-08T04:13:24Z sfcb-ProviderManager[2268449]: handleSigChld:170806080 provider terminated, pid=2268485, exit=0 signal=6

Note:The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.7

Cause

SFCB runs out of memory due to memory leaks in sfcb vmware-base process.

Resolution

This issue is resolved in vSphere ESXi 6.7 Patch  ESXi 670-202011002

Workaround:

• To workaround the issue disable CIM by running the following command:
  esxcli system wbem set -e 0
  
  To confirm it is disabled, run: 
  esxcli system wbem get
  
     Authorization Model: password
     CIMObject Manager PID: 0
     Enabled: false
     Enabled Running SSLProtocols:
     Enabled SSLProtocols:
     Enabled System SSLProtocols: tlsv1.2
     Loglevel: debug
     Port: 5989
     Service Location Protocol PID: 0
     WSManagement PID: 0
     WSManagement Service: true

Additional Information

Impact/Risks:
It will stop hardware monitoring.