Configure alarm for messages of vSAN device detected suspended I/Os with VMkernel Observations

Products

• VMware vSAN 7.x
• VMware vSAN 8.x
• VMware vCenter Server 8.0

Issue/Introduction

When a vSAN host detects that a device has suspended I/Os, an event is reported in the /var/run/log/vobd.log:

--/var/run/log/vobd.log

YYYY-MM-DDTHH:MM:SS In(14) vobd[#######]:  [vSANCorrelator] #######us: [esx.problem.vob.vsan.lsom.stuckiotimeout] vSAN device ########-####-####-####-############ detected I/O timeout error. This may lead to stuck I/O.
YYYY-MM-DDTHH:MM:SS In(14) vobd[#######]:  [vSANCorrelator] #######us: [esx.problem.vob.vsan.lsom.stuckiooffline] vSAN device ########-####-####-####-############ detected stuck I/O error. Marking the device as offline.

Environment

vSAN 8.x

Cause

If it's plan to monitor this event with vCenter alarm, it could be added by manually.

Resolution

Follow the steps mentioned below to set the alarm manually:

1. Navigate to vCenter Server -> Configure -> Alarm Definitions

2. Click Add

3. Input alarm name "vSAN device detected suspended I/Os".

4. Choose the "Target type" as Hosts and click Next

5. In the "Alarm Rule1" page, Input "esx.problem.vob.vsan.lsom.stuckiotimeout" as the trigger condition.

6. Select severity as "Show as Warning"

7. Optionally, the Send email notifications or Send SNMP traps or Run script could be enabled.

8. Click Next

9. In the Review page, verify if the alarm rules is set correctly and make sure the "Enable this alarm" option is enabled.

10. Click CREATE.

Additional Information

It's suggest to refer follow KB to handle the issue of vSAN stuck I/O.

How to handle lost or stuck I/O on a host in vSAN cluster
https://knowledge.broadcom.com/external/article/326885/