vSAN disk format upgrade gets stuck at 97% and eventually fails.

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• After upgrading vSphere from 7.x to 8.x, the vSAN disk format version upgrade from 15 to 20 is failing with the below error:

"msg": "vSAN disk format upgrade failed for cluster cluster! Error: (vim.fault.VsanFault) {\n   dynamicType = <unset>,\n   dynamicProperty = (vmodl.DynamicProperty) [],\n   msg = 'General vSAN error.',\n   faultCause = <unset>,\n   faultMessage = (vmodl.LocalizableMessage) [\n      (vmodl.LocalizableMessage) {\n         dynamicType = <unset>,\n         dynamicProperty = (vmodl.DynamicProperty) [],\n         key = 'com.vmware.vsan.diskconversion.msg.objectconversionerror',\n         arg = (vmodl.KeyAnyValue) [\n            (vmodl.KeyAnyValue) {\n               dynamicType = <unset>,\n               dynamicProperty = (vmodl.DynamicProperty) [],\n               key = 'message',\n               value = 'uuid 9fe24566-126f-8965-c9f6-xxxxxxxxxxxx: Error while converting object.;'\n            }\n         ],\n         message = 'Failed to convert object(s) on vSAN.'\n      }\n   ]\n}",

• Checking the object in the error message, it belongs to one of the running VMs:

/usr/lib/vmware/osfs/bin] ./objtool getAttr -u 9fe24566-126f-8965-c9f6-xxxxxxxxxxx -bypassDom
Object Attributes --

UUID:9fe24566-126f-8965-c9f6-xxxxxxxxxxxx

Object type:vsan

Object size:214748364800

Policy:((\"stripeWidth\" i1) (\"cacheReservation\" i0) (\"proportionalCapacity\" i0) (\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i0) (\"spbmProfileId\" \"07481fc6-9e72-48cc-985b-xxxxxxxxxxxx\") (\"spbmProfileGenerationNumber\" l+2) (\"replicaPreference\" \"Performance\") (\"iopsLimit\" i0) (\"checksumDisabled\" i0) (\"subFailuresToTolerate\" i1) (\"spbmProfileName\" \"VM Stretch Cluster Storage Profile - FTT 1\") (\"locality\" \"None\"))

Object class:vdisk

Object capabilities:NONE

Object path:/vmfs/volumes/vsan:52eca4a14bf5b3f1-xxxxxxxxxxxxxxxx/9de24566-4432-b74b-bc42-xxxxxxxxxxxx/edge.vmdk

Group uuid:9de24566-4432-b74b-bc42-xxxxxxxxxxxx

• The disk management was showing below message:

• Upon clicking upgrade, the process is stuck at 97%

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Cause

The issue occurs if the object has an 'objectDeleted' flag set to 1, due to which the disk format upgrade will fail.

The cmmds dump will show the state of the object is marked for deletion:

[e-mail]_extracted/var/run/log/cmmdsTimeMachineDump.19:2025-07-05T04:06:17.393Z In(14) cmmdsTimeMachineDump[2099234]: id: 749343276800, 9fe24566-126f-8965-c9f6-xxxxxxxxxx,13,17928,6808cb17-4ba1-64a2-61ef-xxxxxxxxxxc,18,{"type": "Configuration", "attributes": {"CSN": 20715, "SCSN": 20510, "addressSpace": 214748364800, "scrubStartTime": 1751635551002428, "objectDeleted": 1, "objectVersion": 15, "highestDiskVersion": 15, "muxGroup": 652138611503624208, "groupUuid": "9de24566-4432-b74b-bc42-xxxxxxxxxxxx", "raidFact": 400, "subClusterUUID": "52eca4a1-4bf5-b3f1-b0f4-xxxxxxxxxxxx", "scrubEta": 791916, "compositeUuid": "9fe24566-126f-8965-c9f6-xxxxxxxxxxxxxxx"}, "child-1": {"type": "RAID_1", "attributes": {"scope": 3}, "child-1": {"type": "RAID_1", "attributes": {"scope": 2}, "child-1": {"type": "Component", "attributes": {"capacity": [0, 214748364800], "addressSpace": 214748364800, "componentState": 5, "componentStateTS": 1749161

Resolution

Verify if the objectDeleted parameter is set to 1 by the below command:

cmmds-tool find -f python>>/tmp/cmmdsdump.txt

If the parameter is marked to 1, consult the Broadcom support for assistance on how to proceed further.