Corruption seen on VMFS datastore due to overwritten data
search cancel

Corruption seen on VMFS datastore due to overwritten data

book

Article ID: 431052

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptom:

  • vMotion operation to/from corrupt datastore remains stuck.

  • Datastore is showing up as inaccessible on vCenter.

Environment

VMware vSphere ESXi 

Cause

  • This often happens if a disk was re-initialized by another OS (like Windows or Linux) or if a storage controller mistakenly wrote a different partition style over an existing VMFS volume. In below example, X.MSDOS5.0 and FAT16 filesystem can be seen. VMFS does not use FAT16.  The presence of a FAT16 file system header within the same space where VMFS metadata is expected confirms that an external process or utility (likely a non-ESXi OS) attempted to format or "initialize" this LUN, overwriting the VMFS volume's "Lock" and "Heartbeat" regions.

    Example -

    The hexdump output clearly shows that LVM header (00200000) and VMFS header (01400000) contain overwritten data, which happens due to external factors. 

Command: hexdump -C /vmfs/devices/disks/naa.################################ | less



  • In /var/run/log/vobd.log of host, we can see below corruption entries -

    2026-01-17T20:31:25.485Z: [vmfsCorrelator] 14387324538201us: [vob.vmfs.resource.corruptondisk] Volume 6290cc69-1aa947e8-1e67-############("VMFS_Datastore") might be damaged on the disk. Resource cluster metadata corruption has been detected.
    2026-01-17T20:31:25.486Z: [vmfsCorrelator] 14388611598536us: [esx.problem.vmfs.resource.corruptondisk] 6290cc69-1aa947e8-1e67-############ VMFS_Datastore

Resolution

Note: While running VOMA advance fix, if the corruption is too severe, VOMA can automatically delete the corrupt files.

  • Run VOMA to validate if the corruption issue can be fixed.

  • Validate the logs again post running VOMA, if the corruption events are still seen then restore the VMs on the corrupt volume from backup.

Additional Information

Using vSphere On-disk Metadata Analyzer (VOMA) to check VMFS metadata consistency

Capture VOMA Dump for VMFS Datastore

Checking Metadata Consistency with vSphere On-disk Metadata Analyzer

OSDATA volume has been detected corrupted