IPMI logs in ESX/ESXi 4.x and ESXi 5.0 consume excessive disk space after a BMC reset or firmware update

Products

VMware vSphere ESXi

Issue/Introduction

This article describes a specific issue. If you experience all of these symptoms, consult the sections below. If you experience some, but not all, of these symptoms, your issue is not related to this article. Search for your symptoms in the Knowledge Base or Open a Support Request.

Symptoms:

The ESX/ESXi host appears to be in a Not Responding state
The host is unreachable in vCenter Server or when using the vSphere Client
The sfcb-vmware_raw or sfcbd process is consuming more than 70MB of memory
The size of the /var/log/ipmi/0/sel file is more than 50MB
The /var/log/ipmi/0/sel file contains the log spew:

System Boot:
The size of the /var/log/ipmi/0/sel.raw file is more than 170MB
The /var/log/ipmi/0/sel.raw file contains the log spew:

efef ef ef ef ef ef ef ef ef ef ef ef ef ef ef
efef ef ef ef ef ef ef ef ef ef ef ef ef ef ef
The /var/log/messages log file contains entries similar to:

sfcb-vmware_raw[nnnn]: IpmiIfcSelReadNew: failed call to IpmiIfcSelReadEntry cc = 0xff
vCenter Server reports the error:

vCenter does not support hosts of this type
When adding or removing a port group from a VSS or when updating the VLAN ID of the port group, you may see the error:

Call "HostNetworkSystem.UpdatePortGroup" for object "networkSystem-####" on vCenter Server "<vCenter FQDN>" failed.
Operation failed, diagnostics report: Sysinfo error on operation returned status : Not found. Please see the VMkernel log for detailed error information

Environment

VMware ESX 4.0.x
VMware ESXi 4.0.x Embedded
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 5.0
VMware ESX 4.1.x
VMware ESXi 4.1.x Embedded
VMware ESXi 4.0.x Installable

Cause

This issue can occur if the CIM and IPMI components in ESX/ESXi 4.x lose connectivity with the hardware Baseboard Management Controller. For example, the host may lose connectivity when the BMC firmware is updated or reset.

Resolution

Solution

This issue is resolved by vSphere 4.1 Update 2, released 2011-10-27. For more information, see the release notes.

Short-term workaround

To workaround this issue in the short-term:

Note: Only use this workaround until you can upgrade to vSphere 4.1 Update 2 or ESXi 5.0 or until you can apply the long-term workaround.

Open a console to the ESX or ESXi host. For more information, see Unable to connect to an ESX host using Secure Shell (SSH) (1003807) or Using Tech Support Mode in ESXi 4.1 (1017910).
Delete the sel and sel.raw files using the commands:

rm /var/log/ipmi/0/sel
rm /var/log/ipmi/0/sel.raw
Restart the sfdbd service on the host using the command:

/etc/init.d/sfcbd-watchdog restart

Note: You do not need to reboot the host for the changes to take effect.

Long-term workaround

To workaround this issue in long-term:

Open a console to the ESX/ESXi host.

For more information, see Unable to connect to an ESX host using Secure Shell (SSH) (1003807) or Using Tech Support Mode in ESXi 4.1 (1017910).
Delete the sel and sel.raw files using the commands:

rm /var/log/ipmi/0/sel
rm /var/log/ipmi/0/sel.raw
Disable IPMI or CIM using one of the options:

Note: After you disable CIM/IPMI, the hardware monitoring information provided by CIM/IPMI is not available.
- Disable IPMI in the VMkernel boot options.
  
  Set the VMKernel.Boot.ipmiEnabled option to 0 or Unchecked. For more information, see Configuring advanced options for ESXi/ESX (1038578).
- Disable the CIM agent on the host. For more information, see How to disable the CIM agent on the ESX/ESXi host (1025757).
Reboot the host for the changes to take effect.

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box

Configuring advanced options for ESXi/ESX
BMC のリセットまたはファームウェアの更新後、ESX/ESXi 4.x および ESXi 5.0 での IPMI ログがディスク領域を過剰に消費する
BMC 重置或固件更新后 ESX/ESXi 4.x 和 ESXi 5.0 中的 IPMI 日志占用的磁盘空间过多