Virtual Machines Down with Unknown/Invalid State
search cancel

Virtual Machines Down with Unknown/Invalid State

book

Article ID: 393502

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms :

  • Multiple objects and virtual machines inaccessible 
  • The following examples show objects in an inaccessible state 

        [root@e1:~] esxcli vsan debug object health summary get
        Health Status                                              Number Of Objects
        ---------------------------------------------------------  -----------------
        inaccessible                                                              34

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Cause

  • When reviewing the vSAN cluster status across various hosts, you may encounter hosts in a DECOMMISSIONED state. This can lead to data unavailability and result in issues when powering on virtual machines.
  • This can be verified on each host using the command below

    [root@e3:~] esxcli vsan cluster get
   Cluster Information
   Enabled: true
   Current Local Time: 2025-04-08T00:56:44Z
   Local Node UUID: 60374a##-5cce-95##-11##-############
   Local Node Type: NORMAL
   Local Node State: BACKUP
   Local Node Health State: HEALTHY
   Sub-Cluster Master UUID: 60374c##-9c##37##-bd##############
   Sub-Cluster Backup UUID: 60374a##-5c##-95##-11##-############
   Sub-Cluster UUID: 52fca5##-5f##a6##-a8##-############
   Sub-Cluster Membership Entry Revision: 3
   Sub-Cluster Member Count: 4
   Sub-Cluster Member UUIDs: 60374c##-9c##-37##-bd##-############, 60374###-5c##-95##-1140-############, 603749##-84##-dc##-27##-############, 60374a##-7d##-7a##-fb4a-############
   Sub-Cluster Member HostNames: e2, e3, e1 ,e4
   Sub-Cluster Membership UUID: 8df9f3##-30##-b29##-8d##-############
   Unicast Mode Enabled: true
   Maintenance Mode State: ON

  • In the example above, the host is in the ENTERING maintenance mode state, which indicates data unavailability for the virtual machines

Resolution

  • Fix the maintenance mode issues with the nodes.
  • To resolve the maintenance mode issue on the nodes, you can cancel the maintenance mode by using the following command "esxcli vsan maintenance mode cancel"

Additional Information

  • Please see below KB as a reference 

         vSAN objects unhealthy due to DECOM state