When attempting to remove a host/ cluster or domain, the tasks fails when VMs are attached on the cluster

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• On VMware Cloud Foundation 3.9.0, if workloads are deployed and virtual machines are already running in it, add host operations will fail. This is due to the Port Group migration from NVDS to VDS during network migration.
• Similar messages to below are seen in/var/log/vmware/vcf/domainmanager/domainmanager.log:

2020-01-16T12:52:01.774+0000 ERROR [3a8437e15cc04484,8329] [c.v.v.c.f.p.n.a.DeleteTransportNodes,pool-2-thread-17]  TRANSPORT_NODE_DELETION_TASK_FAILED
java.lang.RuntimeException: Cannot execute request
 at com.vmware.vapi.internal.protocol.client.rest.DefaultRequestExecutorFactory$DefaultHttpResponseHandler.onError(DefaultRequestExecutorFactory.java:94)
 at com.vmware.vapi.internal.protocol.client.rpc.http.ApacheClientRestTransport.execute(ApacheClientRestTransport.java:79)
 at com.vmware.vapi.internal.protocol.client.rest.DefaultRequestExecutorFactory$DefaultRequestExecutor.execute(DefaultRequestExecutorFactory.java:44)
 at com.vmware.vapi.internal.protocol.client.rest.RestClientApiProvider.invoke(RestClientApiProvider.java:70)
 at com.vmware.vapi.internal.bindings.Stub.invoke(Stub.java:228)
 at com.vmware.vapi.internal.bindings.Stub.invoke(Stub.java:209)
 at com.vmware.vapi.internal.bindings.Stub.invokeMethodAsync(Stub.java:173)
 at com.vmware.vapi.internal.bindings.Stub.invokeMethod(Stub.java:141)
 at com.vmware.nsx.TransportNodesStub.delete(TransportNodesStub.java:85)
 at com.vmware.nsx.TransportNodesStub.delete(TransportNodesStub.java:75)
 at com.vmware.vcf.common.fsm.plugins.nsxt.service.connection.api.nsxt.TransportNodeOperations.delete(TransportNodeOperations.java:56)
 at com.vmware.vcf.common.fsm.plugins.nsxt.service.connection.api.NsxtManagerTransportNodeOperations.deleteTransportNode(NsxtManagerTransportNodeOperations.java:143)
 at com.vmware.vcf.common.fsm.plugins.nsxt.action.DeleteTransportNodes.execute(DeleteTransportNodes.java:161)
 at com.vmware.vcf.common.fsm.plugins.nsxt.action.DeleteTransportNodes.execute(DeleteTransportNodes.java:36)
 Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
 ... 32 common frames omitted
Caused by: java.net.SocketTimeoutException: Read timed out
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
 at java.net.SocketInputStream.read(SocketInputStream.java:171)
 at java.net.SocketInputStream.read(SocketInputStream.java:141)
 at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
 
-> on auto re-trial update is failed
Caused by: com.vmware.vapi.std.errors.InvalidRequest: InvalidRequest (com.vmware.vapi.std.errors.invalid_request) => {
    messages = [],
    data = struct => {error_message=TransportNode [TransportNode/f6d55ade-3dce-416f-8389-daafbd969cc1] can not be updated or deleted while migrating ESX vmk interface 8832c15d-9cd9-f025-afc2-98039b9feb16/HAMS019002.vmx to [DE-HAM01-WLD01-HAMS010029-HAM01COM02-vds01-management]., httpStatus=BAD_REQUEST, error_code=8225, module_name=NsxSwitching service}
}
 at com.vmware.vapi.std.errors.InvalidRequest._newInstance(InvalidRequest.java:163)

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment

Environment

VMware Cloud Foundation 3.9.x

Resolution

This is a known issue affecting VMware Cloud Foundation. This issue is resolved in VMware Cloud Foundation 3.9.1.

Workaround:
A patch has been made available for VMware Cloud Foundation 3.9.0. Previous versions are not supported.

Please contact VMware Global Support for assistance with this issue, referencing this Knowledge Base article.