vSAN Skyline Health Score Was Lowered

Products

• VMware vSAN

Issue/Introduction

The administrator found a drop in the vSAN Skyline Health score history which later returned to the expected level.

Environment

vSAN 8.x

vSAN 9.x

Cause

Physical network issues led to the problems causing the health score drops.

An initial drop to 90% healthy was caused by two hosts being unable to ping each other on the vSAN network.

A further drop ins core to 59% was caused by inaccessible objects detected as the cluster leader node lost communication to other hosts due to network latency, and the cluster reformed with the backup as the new leader.

Leader:

2025-07-15T14:06:28.231Z cpu44:2099213)CMMDS: LeaderUpdateMeanRTLatency:12333: Throttled: 5202fc80-06dd-####-####-############: High RT latency. Node 67803d98-55a0-1318-23e5-e4434ba00a10, RT latency 4774(ms). Mean RT latency 183(ms)
2025-07-15T14:06:28.310Z cpu50:2099213)CMMDS: LeaderLostBackup:568: 5202fc80-06dd-####-####-############: Leader Failover: MUUID 64607668-6479-####-####-############ old 5bbff467-3acb-####-####-############

Others:

2025-07-15T14:06:55.312Z cpu46:2099213)CMMDS: CMMDSLogStateTransition:1917: 5202fc80-06dd-####-####-#############: Transitioning(6245caba-8778-####-####-############) from Discovery to Rejoin: (Reason: Found a leader node)

Resolution

The conditions self resolved without administrator actions.

Additional Information

As the problems originated from the physical network, the network administrator or vendor should be engaged for full cause identification.