Host goes to non-responsive state - "IPMI SEL unavailable"

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Host goes to non-responsive state & shows as disconnected from vCenter. To get the host back online, need to reboot the host.
• In Hostd.log, there might be entry for "hostd detected to be non-responsive". Below entries will be found in hostd.log before & after host going into non-responsive state.

 

Hostd.log

2019-06-11T03:42:51.329Z verbose hostd[11403B70] [Originator@6876 sub=PropertyProvider opID=5aca18f0 user=root] RecordOp ASSIGN: info, haTask--vim.AuthorizationManager.retrieveAllPermissions-156628558. Applied change to temp map.
count_events: starting communication with bmc over ipmi driver
count_events: GET_SEL_REPO_INFO returned {version: 0x51, count 288, free 11776,add_stamp 1551799996, erase_stamp 1438618649 op_support 2}
IPMI SEL sync took 0 seconds 0 sel records, last 288
2019-06-11T03:43:13.066Z error hostd[11403B70] [Originator@6876 sub=Cimsvc] IPMI SEL unavailable

The above entry for IPMI SEL unavailable will be multiple times before & after the host getting unresponsive.

In some environment there won't be any specific entries for "hostd detected to be non-responsive" in hostd.log.

In vmkernel.log, you might see the log snippet "hostd detected to be non-responsive" but it's not always holds true for each & every VMware environment.

Environment

VMware vSphere ESXi 6.5

Cause

This is a known issue started with vSphere 6.5 U2 where excessive KCS traffic causing BMC instability/restart from watchdog.

Resolution

 

This issue is resolved in ESXi 6.5 Update 3, available at Customer Connect.

 

Additional Information

VMware Skyline Health Diagnostics for vSphere - FAQ

IPMI (Intelligent Platform Management Interface) is a specification defining a set of interfaces used for Hardware Platform Management.

Hardware platform management consists of the management of chassis power states and monitoring aspects of devices such as temperature, voltage, fan speed, and other.

IPMI defines a device known as a BMC (Baseboard Management Controller), that acts as central point for the management and monitoring of all IPMI related information in a system. IPMI offers operating systems a variety of interfaces such as KCS (Keyboard Controller Style) to communicate with a BMC.

Impact/Risks:

Host goes to non-responsive state & gets disconnected from vCenter server.