All the objects using the vSAN default policy (FTT=1) are stuck in "reduced-availability-with-no-rebuild" status.
search cancel

All the objects using the vSAN default policy (FTT=1) are stuck in "reduced-availability-with-no-rebuild" status.

book

Article ID: 393984

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

  • The customer is using a vSAN OSA cluster, or configured a new vSAN OSA cluster, and all the objects in the cluster using the FTT=1 vSAN default policy are showing "reduced-availability-with-no-rebuild" status.

  • All the ESXi hosts are healthy and contributing to the vSAN cluster; there are no issues with the disks on any of the ESXi hosts.

  • The virtual objects tab shows the objects in a "Reduced availability with no rebuild" Object State with a blank Storage Policy.


  • When the user runs the "esxcli vsan debug object health summary get" command they see that all the objects are in the "reduced-availability-with-no-rebuild" state.

Health Status                                              Number Of Objects
---------------------------------------------------------  -----------------
remoteAccessible                                                           0
inaccessible                                                               0
reduced-availability-with-no-rebuild                                     234
reduced-availability-with-no-rebuild-delay-timer                           0
reducedavailabilitywithpolicypending                                       0
reducedavailabilitywithpolicypendingfailed                                 0
reduced-availability-with-active-rebuild                                   0
reducedavailabilitywithpausedrebuild                                       0
data-move                                                                  0
nonavailability-related-reconfig                                           0
nonavailabilityrelatedincompliancewithpolicypending                        0
nonavailabilityrelatedincompliancewithpolicypendingfailed                  0
nonavailability-related-incompliance                                       0
nonavailabilityrelatedincompliancewithpausedrebuild                        0
healthy                                                                    0

Environment

VMware vSAN 7.0.x

VMware vSAN 8.0.x

Cause

  • There is only one fault domain available, and this is the reason why all the objects are stuck in the "reduced-availability-with-no-rebuild" status. To apply the vSAN default policy, the user needs three or more fault domains to satisfy the FTT=1 policy (if using Raid 5 or 6 policies the requirement becomes 4 or 6 minimum fault domains respectively).

  • In a vSAN Standard (three or more node) cluster, all the objects should be healthy if all the ESXi nodes in the vSAN cluster are functioning properly and contributing to the storage. 

  • When the user checks the clom logs (/var/log/clomd.log), they see the following excerpts:

2025-04-07T07:02:31.034Z error clomd[2099882] [Originator@6876 opID=1804576233] CLOM_CheckClusterResourcesForPolicy: Not enough Upper FD's available. Available: 1, needed: 3
2025-04-07T07:02:31.034Z error clomd[2099882] [Originator@6876 opID=1804576233] CLOM_GenerateObjectConfig: Cluster doesn't have resources for the current iteration: objVersion: 15 replicas: 1, stripes: 1
2025-04-07T07:02:31.034Z error clomd[2099882] [Originator@6876 opID=1804576233] CLOMGenerateNewConfig: Failed to generate a configuration: Not found
2025-04-07T07:02:31.034Z error clomd[2099882] [Originator@6876 opID=1804576233] CLOM_Diagnose: There are currently 1 usable fault domains. The operation requires 2 more usable fault domains.

  • When the user checks for the fault domain, they notice that all the ESXi hosts are in a single fault domain. The user can check the fault domain information via the following steps.

    • Navigate to the vSAN cluster.
    • Click the Configure tab.
    • Under vSAN, click Fault Domains.

Resolution

  • Remove the existing fault domain, which contains all ESXi hosts. After removing the existing single fault domain, all the ESXi hosts will reside in their own single-host fault domain as Standalone Hosts, which will satisfy the minimum of three fault domains, and the objects will resync to satisfy the FTT=1 policy. Once resync completes the objects will be healthy.

Additional Information