Symptoms:
Some of these symptoms can occur (if not all):
Cluster removal workflow fails with error logs from domainmanager
2023-04-04T09:13:41.396+0000 ERROR [vcf_dm,880ce110b012d15c,9468] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-4] [HSPQ4K] VSPHERE_FAILED_TO_REMOVE_VSAN_POLICY Failed to remove VSAN policy cluster02 vSAN Storage Policy with description vSAN Storage Policy for cluster cluster02
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to remove VSAN policy cluster02 vSAN Storage Policy with description vSAN Storage Policy for cluster cluster02
at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVsanStoragePolicyAction.execute(RemoveVsanStoragePolicyAction.java:104)
at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVsanStoragePolicyAction.execute(RemoveVsanStoragePolicyAction.java:41)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
at jdk.internal.reflect.GeneratedMethodAccessor747.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:88)
at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:73)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:64)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: (pbm.fault.ResourceInUse) {
faultCause = null,
faultMessage = null,
type = null,
name = ########-####-####-####-########8a21
}
The storage policy profile which the workflow was trying to delete was still associated with few unknown entities .
To resolve the issue, please follow the below mentioned steps to manually remove the storage policy from vCenter:
Example: From above logs wld02-cluster02 vSAN Storage Policy is causing the issue (Remove the policy)