[VMC on AWS] Failed to remove host. ESX delete task xxxxxxxxxxxxxx cannot proceed because the workload policy on cluster xxxxxxxxxx, sddc xxxxxxxxxxxxxx does not support x hosts.

Products

• VMware Cloud on AWS

Issue/Introduction

The purpose of this article is to understand the reason for the failure of the remove host task.

Symptoms:

While attempting to remove a host, the task fails with the error:
'Failed to remove host. ESX delete task xxxxxxxxxxxxxx cannot proceed because the workload policy on cluster xxxxxxxxxx, sddc 3b76xxxxxxxxxxx does not support 5 hosts.'  For an FTT of 1 with RAID-5/6, a minimum of 4 hosts are required. For an FTT of 2 with RAID-1, a minimum of 5 hosts are required. For an FTT of 2 with RAID-6, a minimum of 6 hosts are required.

Cause

The configured storage policy should be the reason of the failure of the remove host task.

Resolution

• The error message states that the configured storage policy does not allow the deletion of the ESX host.

• As a first step, verify the storage policies in use in the cluster object.

• Next, ensure that the minimum host requirements are met before proceeding to remove a host.

• The reason for this will be the RAID configurations, FTT and the minimum ESX requirements. For detailed explanation, please refer the vSAN Policies.

• If the existing storage policy configured is RAID-6 with FTT-2, then minimum hosts will have to be 6.

• Any attempt to delete the 6th host will lead to failure in removal of host.

• If the existing storage policy is a custom storage policy, then the custom storage policy can be changed to something like RAID-1 FTT=2 or any other storage policy as desired and then re-attempt the host removal.

• There might be an increase in vSAN latency times as vSAN rebalances all the data across the cluster but there should not be any hard impact with Networking, CPU, memory.

• It is suggested to create a new custom storage policy that is RAID-1 FTT=1 or any other storage policy that is not RAID-6 and then proceed to move batches of 5-10 VMs at a time from the RAID-6 custom policy to the RAID-1(new) custom policy and then allow a few hours in between to allow vSAN to resync the data. This should allow for minimal impact in latency, if any. 

• Also, doing these transitions outside of peak-hours would also help in ensuring that there are no production impacts.