vSAN cluster fails to create virtual machines with error “Cannot complete file creation operation. Failed to place witnesses” after converting stretched cluster to standard single-site cluster

Products

• VMware vSAN

Issue/Introduction

In a vSAN environment that was previously configured as a stretched cluster, virtual machine creation fails after the witness appliance is deleted and the cluster is converted into a standard single-site cluster.

The following error appears when attempting to create a VM or deploy an object:

Cannot complete file creation operation. Failed to place witnesses. There are currently 0 usable disks and 1 more usable disks are needed in witness node. Failed to create object.

Environment

• VMware vSAN 7.x / 8.x / 9.x

• Cluster originally configured as stretched

• Witness appliance VM later removed manually

• Cluster reconfigured to standard single-site cluster.

Cause

The issue occurred because the cluster was converted from a stretched configuration to a single-site configuration using an unsupported or incomplete procedure.

As a result, a stale witness record remained in the vSAN unicastagent configuration, causing vSAN to continue attempting to place witness components on a non-existent node.

Resolution

1. Verify current unicast agents on each ESXi host
   

   Connect to each ESXi host via ssh and verify current unicastagents by this command,

   esxcli vsan cluster unicastagent list

    

2. One of the entries shows IsWitness: 1 and its address corresponds to the deleted witness appliance. (The following is a partial output of above command. Only key columns such as NodeUuid, IsWitness, Supports Unicast, and IP Address are shown here for brevity.)

   NodeUuid                              IsWitness  Supports Unicast  IP Address   
   ------------------------------------  ---------  ----------------  -----------  
   <NODE UUID>                                   0              true  <NODE IP>  
   <NODE UUID>                                   0              true  <NODE IP>  
   <NODE UUID>                                   1              true  <OLD WITNESS IP>   <=== This is the entry of old witness

    

3. Remove the obsolete witness record from all hosts

   esxcli vsan cluster unicastagent remove --addr <OLD WITNESS IP>

   
   Replace <OLD WITNESS IP> with the IP address of the old witness appliance.

    

4. Confirm that no unicast entries remain marked IsWitness = 1

   esxcli vsan cluster unicastagent list

5. Re-attempt VM creation
   
   Once all stale witness entries are removed, vSAN updates its cluster membership and new virtual machine creation completes successfully.