Unexpected objects or inaccessible objects appear after adding disk(s) to vSAN disk group or adding host back to the cluster after objects have been deleted
Article ID: 373216
Updated On:
Products
Issue/Introduction
After adding a disk back to an existing vSAN disk group, or adding a host back to the cluster that has been out of the cluster for a period of time, one or more unexpected unassociated or inaccessible objects are reported.
Investigation of the object(s) shows that:
1. The object found is not expected to be in the current cluster (typically for an object that had been deleted previously).
2. There is one or more components active on the disk that was added to the disk group, but no other components for the object(s).
Note: If multiple disks were readded there may be components from each of the disks added.
Example object layout from esxcli vsan debug object list -u <object UUID>
/bin/esxcli vsan debug object list -u 10274465-50e7-9602-####-############
Object UUID: 10274465-50e7-9602-####-############
Version: 19
Health: inaccessible - Lost data availability.(APD)
Owner: hostname.example.com
Size: 0.00 GB
Used: 30.27 GB
Used 4K Blocks: 0.00 GB
Policy:
Configuration:
RAID_1
RAID_0
Component: cd34d465-dcf0-f528-9502-############
Component State: ABSENT, Address Space(B): 273804165120 (255.00GB), Disk UUID: 528e554b-c38d-b229-881a-############, Disk Name: N/A
Votes: 1, Host UUID: None
Component: cd34d465-ccca-f728-8f96-############
Component State: ABSENT, Address Space(B): 273804165120 (255.00GB), Disk UUID: 52741a7e-8cbf-f69b-ee21-############, Disk Name: N/A
Votes: 1, Host UUID: None
Component: bd42d565-4c5e-5d34-f298-############
Component State: ABSENT, CSN: STALE (7168!=7171), Address Space(B): 273804165120 (255.00GB), Disk UUID: 523bad59-3167-feb4-28a7-############, Disk Name: N/A
Votes: 1, Host UUID: None
RAID_0
Component: f356dc65-627b-1345-cd50-############
Component State: ABSENT, Address Space(B): 273804165120 (255.00GB), Disk UUID: 52cdc4d3-6883-c5ac-26cc-############, Disk Name: N/A
Votes: 1, Host UUID: None
Component: cd34d465-5ca0-fd28-807b-############
Component State: ABSENT, Address Space(B): 273804165120 (255.00GB), Disk UUID: 520c77d1-63d9-b0b6-45b1-############, Disk Name: N/A
Votes: 1, Host UUID: None
Component: cd34d465-5c8c-ff28-741b-############
Component State: ACTIVE, Address Space(B): 273804165120 (255.00GB), Disk UUID: 529e2520-927f-b88f-6584-############, Disk Name: eui.01000000000000028ce38ee21b421c00:2
Votes: 1, Capacity Used(B): 7115324245 (6.63GB), Physical Capacity Used(B): 32497074176 (30.27GB), Total 4K Blocks Used(B): 0 (0.00GB), Host Name: hostname.example.com
Type: vmnamespace
Path: /vmfs/volumes/vsan:5208033423de393b-3################/.vsan.stats (Exists)
Group UUID: 10274465-50e7-9602-####-############
Directory Name: .vsan.stats
Environment
VMware vSAN
Cause
The disk(s) added back to the vSAN disk group were not properly removed from use resulting in legacy data that should have been deleted.
The host was out of use in the cluster (extended maintenance mode with the No action option, powered off, failed in some way so vSAN disks not in use) and did not receive the delete operation for the object. This results in remaining component(s) for that object which vSAN then reads and finds the remaining expected components for that object missing.
Resolution
To avoid this situation ensure all disk or host removals are performed with the proper decommission procedures when possible. See the following resources for processes:
How to remove a disk from a vSAN disk group/host
Permanently Decommissioning a node from a vSAN Cluster
Place a Member of vSAN Cluster in Maintenance Mode
If you find yourself in this situation then follow the steps below:
- Verify the objects seen are not expected to be present or in use.
- Open a case with vSAN Support to assist with deleting the objects. Creating and managing Broadcom support cases
Feedback
