Hardware health alarm repeatedly triggered.

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

The hardware health alarm repeatedly transitions between Red and Green.

After a storage failure, despite the disk remaining in a failed state, the health status repeatedly transitions between Green and Red.

vpxd[####]: Event [#######] [#-#] [YYYY-MM-DDTHH:MM:SS] [vim.event.AlarmStatusChangedEvent] [info] [] [##-##] [#######] [Alarm 'Host storage status' on host.sample.com changed from Red to Green]
vpxd[####]: Event [#######] [#-#] [YYYY-MM-DDTHH:MM:SS] [vim.event.AlarmStatusChangedEvent] [info] [] [##-##] [#######] [Alarm 'Host storage status' on host.sample.com changed from Green to Red]

Cause

The iDRAC firmware was reset at the corresponding time, and it is possible that the sensor information was reset.
This issue may re-occur because the sensor information will be re-acquired until the storage-side problem is resolved.

The following records can be observed in /var/run/log/syslog on the host experiencing the storage failure when alarm with Green status is triggered.

YYYY-MM-DDTHH:MM:SS sfcb-dcism[2103846]: ISM0007 [##### EventID="#####" EventCategory="Audit" EventSeverity="warn" IsPastEvent="false" language="en-US"] The iDRAC Service Module communication with iDRAC has ended.

Resolution

Contact the hardware vendor for details on the reset behavior.