Unexpected objects or inaccessible objects appear after adding disk(s) to vSAN disk group or adding host back to the cluster after objects have been deleted
search cancel

Unexpected objects or inaccessible objects appear after adding disk(s) to vSAN disk group or adding host back to the cluster after objects have been deleted

book

Article ID: 373216

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

After adding a disk back to an existing vSAN disk group, or adding a host back to the cluster that has been out of the cluster for a period of time, one or more unexpected unassociated or inaccessible objects are reported.

Investigation of the object(s) shows that:
     1. The object found is not expected to be in the current cluster (typically for an object that had been deleted previously).
     2. There is one or more components active on the disk that was added to the disk group, but no other components for the object(s).  
Note: If multiple disks were readded there may be components from each of the disks added.

Example object layout from esxcli vsan debug object list -u <object UUID>


 /bin/esxcli vsan debug object list -u 10274465-50e7-9602-####-b83fd2962e36
Object UUID: 10274465-50e7-9602-####-b83fd2962e36
   Version: 19
   Health: inaccessible - Lost data availability.(APD)
   Owner: hostname.example.com
   Size: 0.00 GB
   Used: 30.27 GB
   Used 4K Blocks: 0.00 GB
   Policy:
   Configuration:

         RAID_1
            RAID_0
               Component: cd34d465-dcf0-f528-9502-b83fd2962dce
                 Component State: ABSENT,  Address Space(B): 273804165120 (255.00GB),  Disk UUID: 528e554b-c38d-b229-881a-da998af14e73,  Disk Name: N/A
                 Votes: 1,  Host UUID: None
               Component: cd34d465-ccca-f728-8f96-b83fd2962dce
                 Component State: ABSENT,  Address Space(B): 273804165120 (255.00GB),  Disk UUID: 52741a7e-8cbf-f69b-ee21-4d9805327b1b,  Disk Name: N/A
                 Votes: 1,  Host UUID: None
               Component: bd42d565-4c5e-5d34-f298-b83fd2962dce
                 Component State: ABSENT,  CSN: STALE (7168!=7171),  Address Space(B): 273804165120 (255.00GB),  Disk UUID: 523bad59-3167-feb4-28a7-2281e7a1f3dd,  Disk Name: N/A
                 Votes: 1,  Host UUID: None
            RAID_0
               Component: f356dc65-627b-1345-cd50-b83fd2962dce
                 Component State: ABSENT,  Address Space(B): 273804165120 (255.00GB),  Disk UUID: 52cdc4d3-6883-c5ac-26cc-3c11f8bbe02c,  Disk Name: N/A
                 Votes: 1,  Host UUID: None
               Component: cd34d465-5ca0-fd28-807b-b83fd2962dce
                 Component State: ABSENT,  Address Space(B): 273804165120 (255.00GB),  Disk UUID: 520c77d1-63d9-b0b6-45b1-f282a15a3080,  Disk Name: N/A
                 Votes: 1,  Host UUID: None
               Component: cd34d465-5c8c-ff28-741b-b83fd2962dce
                 Component State: ACTIVE,  Address Space(B): 273804165120 (255.00GB),  Disk UUID: 529e2520-927f-b88f-6584-68c5cd413392,  Disk Name: eui.01000000000000028ce38ee21b421c00:2
                 Votes: 1,  Capacity Used(B): 7115324245 (6.63GB),  Physical Capacity Used(B): 32497074176 (30.27GB),  Total 4K Blocks Used(B): 0 (0.00GB),  Host Name: hostname.example.com

   Type: vmnamespace
   Path: /vmfs/volumes/vsan:5208033423de393b-3####a0ed8d32342/.vsan.stats (Exists)
   Group UUID: 10274465-50e7-9602-####-b83fd2962e36
   Directory Name: .vsan.stats

Environment

VMware vSAN

Cause

The disk(s) added back to the vSAN disk group were not properly removed from use resulting in legacy data that should have been deleted.

The host was out of use in the cluster (extended maintenance mode with the No action option, powered off, failed in some way so vSAN disks not in use) and did not receive the delete operation for the object. This results in remaining component(s) for that object which vSAN then reads and finds the remaining expected components for that object missing.

Resolution

To avoid this situation ensure all disk or host removals are performed with the proper decommission procedures when possible. See the following resources for processes:
How to remove a disk from a vSAN disk group/host
Permanently Decommissioning a node from a vSAN Cluster
Place a Member of vSAN Cluster in Maintenance Mode

If you find yourself in this situation then follow the steps below: