VSAN upgrade failed at the step `Check existing objects on the vSAN cluster step in the VC` during automation upgrade process

search cancel

VSAN upgrade failed at the step `Check existing objects on the vSAN cluster step in the VC` during automation upgrade process

book

Article ID: 419826

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms

When attempting to upgrade a vSAN cluster using the automated lifecycle manager (vLCM) or Update Manager (VUM), the process fails at the pre-check stage.

Error Message: The upgrade task fails in vCenter with the error:

Check existing objects on the vSAN cluster.
Health Status: The vSAN Skyline Health UI reports the cluster is Green/Healthy.
CLI Status: Running esxcli vsan debug object health summary get on hosts indicates all active objects are health

Health Status                                              Number Of Objects
---------------------------------------------------------  -----------------
remoteAccessible                                                           0
inaccessible                                                               0
reduced-availability-with-no-rebuild                                       0
reduced-availability-with-no-rebuild-delay-timer                           0
reducedavailabilitywithpolicypending                                       0
reducedavailabilitywithpolicypendingfailed                                 0
reduced-availability-with-active-rebuild                                   0
reducedavailabilitywithpausedrebuild                                       0
data-move                                                                  0
nonavailability-related-reconfig                                           0
nonavailabilityrelatedincompliancewithpolicypending                        0
nonavailabilityrelatedincompliancewithpolicypendingfailed                  0
nonavailability-related-incompliance                                       0
nonavailabilityrelatedincompliancewithpausedrebuild                        0
healthy                                                                 1403

Logs: In the vsanmgmt.log or vsansystem.log, you may see references to object UUIDs identified as "Inaccessible" or "Unhealthy" despite the UI showing otherwise.

vsanmgmt.all:2025-11-19T04:02:53.908Z Wa(12) vsand[26749977]: [opID=eaeac0dd VsanObjectHelper::GetObjectsPolicyWithRetry] Failed to determine compliance for 6 objects: 508ee665-3a29-####-####-0c42a15ac698, a18ce665-2a9f-####-####-0c42a1cb1ec0, 386dbd65-1483-####-####-0c42a1cb8570, fec7e365-64d9-####-####-0c42a1cb37b8, 8c63e665-52b7-####-####-0c42a11cb326, ea66e665-024a-####-####-0c42a1cb4e58

vsansystem.all:2025-11-18T17:08:04.363Z Wa(164) vsansystem[26071488]: [vSAN@6876 sub=Libs opId=b89a1563-435] Object ea66e665-####-####-9b8a-0c42a1cb4e58 not found in CMMDS with error Not found

vsansystem.all:2025-11-18T17:08:04.363Z Er(163) vsansystem[26071488]: [vSAN@6876 sub=CMMDSUtil opId=b89a1563-435] Failed to check accessibility state for object 'ea66e665-024a-####-####-0c42a1cb4e58': Not found

vsansystem.all:2025-11-18T17:13:04.647Z Wa(164) vsansystem[26071488]: [vSAN@6876 sub=Libs opId=b89a1563-5f9] Object ea66e665-####-####-9b8a-0c42a1cb4e58 not found in CMMDS with error Not found

 8c63e665-52b7-####-####-0c42a11cb326
Version:
  14
Owner:
  00000000-0000-0000-0000-000000000000
Policy:

Used:
  0
Used 4K Blocks:
  0
Size:
  0
Type:
  N/A
Path:
  N/A
Group UUID:
  e255d465-88fe-81e0-b630-0c42a1cb37b8
Directory Name:
  N/A
Configuration:

      RAID_1
         RAID_1
            RAID_D
               Component: 8c63e665-8a3b-####-####-0c42a11cb326
                 Component State: ABSENT,  CSN: STALE (24!=32),  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 525539eb-6025-####-####-9737cc73d112,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####:2
                 Votes: 1,  Host Name: esxi
               Component: 16c1e665-####-####-2320-0c42a162cb18
                 Component State: ACTIVE,  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 52c9ba72-07ef-####-####-4c210d377515,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####:2
                 Votes: 1,  Host Name: esxi
            Component: 8c63e665-d8b5-####-####-0c42a11cb326
              Component State: ABSENT,  CSN: STALE (24!=32),  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 52c7a12d-####-####-5f77-062af3c444ab,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####:2
              Votes: 1,  Host Name: esxi
            Component: 8c63e665-7024-####-####-0c42a11cb326
              Component State: ABSENT,  CSN: STALE (24!=32),  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 5217216f-3e35-####-####-5d2a85ee0aaa,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####:2
              Votes: 1,  Host Name: esxi
         RAID_1
            Component: 8c63e665-5a12-####-####-0c42a11cb326
              Component State: ACTIVE,  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 523ba64b-60dc-####-####-c6a3a5addc4b,  Disk Name: N/A
              Votes: 1,  Host UUID: None
            Component: 8c63e665-c895-####-####-0c42a11cb326
              Component State: ACTIVE,  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 5262509d-bd74-####-####-da7677582835,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####
              Votes: 1,  Host Name: esxi
            Component: 8c63e665-64bf-####-####-0c42a11cb326
              Component State: ACTIVE,  Address Space(B): 107374182400 (100.00GB),  Disk UUID: 5209a6ac-4f06-####-####-f2055442c0ab,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####
              Votes: 1,  Host Name: esxi
      Witness: 8c63e665-8e27-####-####-0c42a11cb326
        Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: 52351774-0c6e-####-####-9aa0b1c18631,  Disk Name: N/A
        Votes: 5,  Host UUID: None
      Witness: 8c63e665-aa2e-####-####-0c42a11cb326
        Component State: ABSENT,  CSN: STALE (24!=32),  Address Space(B): 0 (0.00GB),  Disk UUID: 529efe18-1888-####-####-c98e82059b63,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####
        Votes: 1,  Host Name: esxi
      Witness: 8c63e665-502f-####-####-0c42a11cb326
        Component State: ABSENT,  CSN: STALE (24!=32),  Address Space(B): 0 (0.00GB),  Disk UUID: 52f94f7c-ea44-####-####-e4e2c817230d,  Disk Name: N/A
        Votes: 1,  Host UUID: None
      Witness: 8c63e665-e073-####-####-0c42a11cb326
        Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: 52ed49f9-b5f3-a0ef-5a3c-3bd22a8bd23a,  Disk Name: N/A
        Votes: 1,  Host UUID: None
      Witness: 8c63e665-62ae-####-####-0c42a11cb326
        Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: 527f1a63-5d5f-1482-6f02-374e5eebaee2,  Disk Name: t10.NVMe____Dell_Express_Flash_PM1725b_3.2TB_SFF____####
        Votes: 1,  Host Name: esxi

Environment

vSAN 8.x

Cause

This issue occurs because the vSAN upgrade process performs a strict metadata consistency check that is more sensitive than the standard Health UI.

The upgrade process has detected inaccessible or orphaned objects in the vSAN Datastore.These are usually leftover objects from previous failed deletions, old .vswp (swap) files, or temporary data that the vCenter inventory is no longer tracking.

Resolution

To resolve this issue, you must identify and remove the inaccessible/stale objects.
In order to delete object ,please open a case with Broadcom Technical Support to investigate further.

Feedback

thumb_up Yes

thumb_down No