vSAN RAID_D Components Stuck in Resync State Due to Stale Delta Components in ESA Clusters

Products

• VMware vSAN

Issue/Introduction

Symptoms

• vSAN ESA cluster configured with a RAID-6 (FTT=2) storage policy.
• RAID_D components are observed in an Absent or Stale state.
• The DeltaComponent feature is enabled as confirmed using the following command on the ESXi host:

/> cat /config/VSAN/intOpts/DeltaComponent
Vmkernel Config Option {
   Default value:1           <====
   Min value:0
   Max value:1
   Current value:1          <====
   hidden config option:1
   Description:Whether Delta Component feature is turned on
   Host specific config option:0
   Option update requires reboot:0
   Option update requires maintenance mode:0

 

 

Environment

VMware vSAN 8.x 

Cause

The issue occurs due to stale concatenated (concat) legs in the RAID_D component layout of a vSAN ESA cluster. vSAN attempts to reconcile or heal these components but fails to clean them up due to internal inconsistencies.

The warning Prepare at a lower LSN than LCW in /var/log/vmkernel.log indicates that a prepare operation is being attempted with a Log Sequence Number (LSN) lower than the Last Committed Write (LCW).

2025-05-07T18:19:04.923Z Wa(180) vmkwarning: cpu93:2099759)WARNING: DOM: DOMComponentReconcileComponentProcessEntry:12453: Prepare at a lower LSN than LCW for ########-####-####-###########1  CSN/LSN=76807277688880:3398159051276 vs LCW CSN/LSN=1901:339815905137$
2025-05-07T18:19:05.775Z Wa(180) vmkwarning: cpu63:2099747)WARNING: DOM: DOMComponentReconcileComponentProcessEntry:12453: Prepare at a lower LSN than LCW for ########-####-####-###########2  CSN/LSN=76676826552624:11725810223972 vs LCW CSN/LSN=1863:11725810224$

This inconsistency disrupts object health tracking and prevents vSAN from successfully completing component cleanup. Common causes include:

• Receipt of stale or replayed write operations.
• Object state corruption or timing mismatches during resync.
• Incomplete reconciliation of Delta components introduced by RAID_D layout logic in ESA.

Resolution

Workaround

1. Migrate the VM to a non-vSAN datastore using Storage vMotion. Then, migrate the cloned VM back to vSAN datastore.

2. Clone the virtual machine to create a fresh instance without residual stale metadata.

3. Apply a RAID-1 (FTT=2) policy on the cloned VM to reinitialize component layout.

Additional Information

• Ensure you validate object health via RVC or vsan.obj_status_report before and after migration.
• A support bundle can be collected and analyzed if components remain stuck after the above steps.
• Avoid force-deletion of components, as this can lead to data loss or further inconsistencies.