Remove Cluster failure for VLCM based domain with error: VSPHERE_FAILED_TO_REMOVE_VSAN_POLICY

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

Some of these symptoms can occur (if not all):
Cluster removal workflow fails with error logs from domainmanager
2023-04-04T09:13:41.396+0000 ERROR [vcf_dm,880ce110b012d15c,9468] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-4]  [HSPQ4K] VSPHERE_FAILED_TO_REMOVE_VSAN_POLICY Failed to remove VSAN policy wld02-cluster02 vSAN Storage Policy with description vSAN Storage Policy for cluster wld02-cluster02
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to remove VSAN policy wld02-cluster02 vSAN Storage Policy with description vSAN Storage Policy for cluster wld02-cluster02
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVsanStoragePolicyAction.execute(RemoveVsanStoragePolicyAction.java:104)
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVsanStoragePolicyAction.execute(RemoveVsanStoragePolicyAction.java:41)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
        at jdk.internal.reflect.GeneratedMethodAccessor747.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:88)
        at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:73)
        at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:64)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: (pbm.fault.ResourceInUse) {
   faultCause = null,
   faultMessage = null,
   type = null,
   name = f1216a66-1ba4-4225-b560-19eb2b8a8a21
}

Environment

Vmware Cloud Foundation 4.5.1

Cause

The storage policy profile which the workflow was trying to delete was still associated with few unknown entities .

Resolution

To resolve the issue, please follow the below mentioned steps to manually remove the storage policy from vCenter:

1. From the vSphere Web Client Home, click Policies and Profiles> VM Storage Policies.
2. Click the VM Storage Policies tab.
3. Select the policy to delete and click the Delete icon ().
4. You can select multiple policies.
5. Click Yes to confirm your action.
6. Restart the workflow.

Example: From above logs wld02-cluster02 vSAN Storage Policy is causing the issue (Remove the policy)