Unable to delete inaccessible vSAN object

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Attempting to delete the inaccessible vSAN object from vSAN Skyline Health > Data > Object Health > "Purge Inaccessible VM Swap Objects" (for vm swap objects) or with objtool command completes with no errors, but the inaccessible vSAN object does not get deleted.

• Owner abdication of the object does not change the owner of the object.
  Note: There can be times when the owner does change, but after attempting to delete the object the owner changes back to the original one.

Environment

VMware vSAN 8.x (ESA)

Cause

• The inaccessible vSAN object is unable to be deleted as the components which make it up has reached their max "num prepares" value of 224.

• If this occurs no further operations for the object will proceed, such as deletion.

• /var/run/log/epd.log on ESXi host reports the below for the inaccessible object:

  YYYY-MM-DDTHH:MM:SS No(13) epd[2100587]: EPDGetEntryStalenessForSimplifiedDC: Entry ########-####-####-####-############ is stale
YYYY-MM-DDTHH:MM:SS No(13) epd[2100587]: EPDGetEntryStalenessForSimplifiedDC: Entry ########-####-####-####-############ is stale

• The value for "Num prepares" for an absent component is 224:

  1. Choose an absent component for the inaccessible vSAN object, make a note of it's UUID.

  2. For this component, make a note of the "Disk UUID" & "Host name". To get components of the inaccessible object, run the command: esxcli vsan debug object list -u <inaccessible_object_uuid>

     Example of a component from an inaccessible object hit with this issue:
     Component: <component_uuid>
       Component State: ABSENT,  Address Space(B): ####### (####GB),  Disk UUID: <disk_uuid>,  Disk Name: <disk_name>
       Votes: 2,  Capacity Used(B): 0 (0.00GB),  Physical Capacity Used(B): ##### (###GB),  Total 4K Blocks Used(B): 0 (0.00GB),  Host Name: <hostname>

  3. Log into SSH of this host and run this command to verify the num prepares value: vsish -e get /vmkModules/lsom/lsom2/disks/<disk_uuid>/components/<component_uuid>/info
     Output similar to:
     {

        Partition UUID:<component_uuid>
   Partition type:capacity
   Component pointer:0xxxxxxxxxxxxx
   Component ID:########
   Comp table page Idx:1
   Comp table entry Idx:2
   State:4
   Num prepares:224          <<<<<<<<<<<<<<
   .
   .

Resolution

The fix for this issue is available in ESXi 8.0U2b (24280767).

If the above symptoms and issue match, please contact Broadcom Support to investigate the issue.

 

Note: This situation can also arise for objects which were inaccessible when the ESXi host was on a version below 8.0U2b and the hosts are now on version 8.0U2b or above.

Additional Information

VMs with large-sized vmdks on vSAN ESA may become inaccessible