vSAN VMs Restart Unexpectedly During Single NIC Failure in LACP LAG Due to Timer Mismatch
search cancel

vSAN VMs Restart Unexpectedly During Single NIC Failure in LACP LAG Due to Timer Mismatch

book

Article ID: 445962

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms

  • Multiple virtual machines (VMs) in a vSAN cluster restart unexpectedly.
  • vSphere High Availability (HA) reports that VMs have been restarted following a host isolation event.
  • The issue occurs despite redundant network uplinks being configured in an LACP Link Aggregation Group (LAG).
  • vSAN Skyline Health may report transient "Network Partition" or "Data Availability" warnings.
  • The following log patterns are observed in the ESXi support bundle: 

/var/run/log/vobd.log :

2026-06-24T16:07:25.269Z In(14) vobd[2097812]:  [netCorrelator] 2820463594545us: [vob.net.vmnic.linkstate.up] vmnic vmnic0 link state up

/var/run/log/vmkernel.log :

2026-06-24T16:03:43.346Z In(182) vmkernel: cpu58:2098699)CMMDS: CMMDSAgentlikeCheckNode:3119: 52######-80##-28##-66##-d4##########: Lost contact with leader.

/var/run/log/vsansystem.log :

2026-06-24T16:03:47.660Z In(166) vsansystem[2100167]: [vSAN@6876 sub=VsanSystemProvider opId=CMMDSMembershipUpdate-d4e7] Complete, nodeCount: 1, runtime info: (vim.vsan.host.VsanRuntimeInfo) {

 

Environment

VMware vSAN 8.x

Cause

  • The unexpected restarts are caused by a vSAN network partition that occurs because the redundant link in the LACP bundle fails to take over traffic before the vSAN heartbeat threshold is exceeded.
  • In environments configured with Slow LACP timers (30-second transmission intervals, 90-second timeout), the upstream physical switch may continue to route return traffic to the failed link for up to 90 seconds. Because vSAN declares a network partition after only 10 seconds of lost communication, the ESXi host becomes isolated and triggers a vSphere HA failover long before the physical network convergence is complete. This results in a "black hole" effect where the surviving uplink is unable to receive the traffic necessary to maintain cluster membership.
  • vDS details related to LAG appears as below .

 DVSwitch: vSAN-Cluster-Test
   Flags: S - Device is sending Slow LACPDUs, F - Device is sending fast LACPDUs, A - Device is in active mode, P - Device is in passive mode
   LAG ID: 1#######
   Mode: Active
   Nic List:
            Local Information:
                  Admin Key: 15
                  Flags: SA
                  Oper Key: 1#
                  Port Number: 2
                  Port Priority: 255
                  Port State: ACT,AGG,SYN,COL,DIST,
            Nic: vmnic1
            Partner Information:
                  Age: 00:00:02
                  Device ID: 0#:2#:0#:e#:b#:7#
                  Flags: SA
                  Oper Key: 3####
                  Port Number: 16677
                  Port Priority: 32768
                  Port State: ACT,AGG,SYN,COL,DIST,
            State: Bundled

            Local Information:
                  Admin Key: 15
                  Flags: SA
                  Oper Key: 1#
                  Port Number: 1
                  Port Priority: 255
                  Port State: ACT,AGG,SYN,COL,DIST,
            Nic: vmnic0
            Partner Information:
                  Age: 00:00:18
                  Device ID: 0#:2#:0#:e#:b#:7#
                  Flags: SA
                  Oper Key: 33###
                  Port Number: 293
                  Port Priority: 32768
                  Port State: ACT,AGG,SYN,COL,DIST,
            State: Bundled

Resolution

To resolve this issue, LACP convergence must be optimized to occur faster than the vSAN isolation threshold.

  1. Enable Fast LACP Timers: Configure the LACP bundle on both the vSphere Distributed Switch (vDS) and the physical switch ports to use Fast Timers (1-second intervals, 3-second timeout). This ensures that traffic is redistributed to the surviving uplink within 3 seconds of a failure.
  2. Verify VLAN Consistency: Ensure that the vSAN VLAN is correctly trunked on all physical switch ports participating in the LAG. If the redundant port is missing the VLAN configuration, failover will fail.
  3. Configure Spanning Tree (STP) for PortFast: Set the physical switch ports to PortFast or Edge Port mode. This prevents the surviving uplink from entering a temporary blocking state during network topology changes.
  4. Audit Physical Path: Inspect the physical cabling and intermediate hardware to ensure that a "Link Down" state on the ESXi host is immediately propagated to the physical switch.