VMware Cloud Foundation Remove VI failed in Delete Compute Manager action

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• During remove VI, VCF workflow is deleting all VMs including manually created EDGE VM (Not created by VCF automation).
• Remove VI failed in Delete Compute manager action with below error

2021-07-23T15:36:56.877+0000 ERROR [vcf_dm,27b6306fb7cd2b05,c034] [c.v.v.c.f.p.n.p.a.DeleteComputeManagerAction,dm-exec-18] COMPUTE_MANAGER_DELETION_TASK_FAILED
java.lang.RuntimeException: Retriable operation 'Deleting compute manager: vcenter-vsan.vrack.vsphere.local' failed to complete after 3 retries.
at com.vmware.evo.sddc.common.util.RetriableCallable.call(RetriableCallable.java:186)
at com.vmware.vcf.common.fsm.plugins.nsxt.policy.action.DeleteComputeManagerAction.execute(DeleteComputeManagerAction.java:121)
at com.vmware.vcf.common.fsm.plugins.nsxt.policy.action.DeleteComputeManagerAction.execute(DeleteComputeManagerAction.java:41)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.lambda$static$0(FsmActionState.java:14)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:61)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:158)
at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:143)
at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:399)
at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:519)
at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:123)
at sun.reflect.GeneratedMethodAccessor920.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67)
at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.vmware.vapi.std.errors.InvalidRequest: InvalidRequest (com.vmware.vapi.std.errors.invalid_request) => {
messages = [],
data = struct => {error_message=Compute manager f96f6f88-4ffb-4388-bfd2-10112d346b8f is in use. Please delete nsx managed VM(s) [nsxt-1-edge.vrack.vsphere.local] and then retry deleting compute manager., httpStatus=BAD_REQUEST, error_code=26704, module_name=service-installation-fabric},
errorType = INVALID_REQUEST

Environment

VMware Cloud Foundation 4.x

Resolution

Currently there is no resolution

Workaround:
To workaround this issue, please follow the below steps:

1. Login to NSXT-UI.
2. Identify TIER-1 and TIER-0 routers specific to the Edge clusters deployed on the current domain and delete them respectively.

   Networking > Tier-1 Gateways
   Networking > Tier-0 Gateways

3. Delete the associated Edge Clusters (Fabric > Nodes > Edge Clusters).
4. Delete the associated Edge VMs and wait for the VMs to be deleted (Fabric > Nodes > Edge Transport Nodes).
5. Retry the failed Delete Vi workflow.

Note: It is important to identify the objects being deleted are exactly associated with the Edge cluster deployed on the current domain attempted to be deleted.