Symptoms:
- When performing a test failover to a cloud site the task fails and you see similar messages in the Replication Tasks view in the Cloud Director Availability Portal:
Failed to connect NICs for vm: b661b1a5-6e78-####-####-########839
Unexpected vCloud Director error. [ UI-15f113ff-2b02-####-####-########f7d-5O-15b4de70-7da4-####-####-########728 ] null
Unable to restore the Guest Customization settings of recovery virtual machine 'vm_name'.
Unexpected vCloud Director error. [ UI-15f113ff-2b02-####-####-########f7d-5O-e79a8c52-6587-####-####-########bd8 ] null
- In the /opt/vmware/vcloud-director/logs/vcloud-container-debug.log file on the Cloud Director cell, you see entries similar to:
2020-12-16 11:40:33,232 | ERROR | storage-fabric-activity-pool-1462 | RelocateStoredVmByDatastoreActivity | [Activity Execution] Uncaught Exception during Activity execution. Recent phase: com.vmware.vcloud.fabric.storage.storedVm.impl.RelocateStoredVmByDatastoreActivity$MigratePoweredOnVmAcrossRpAndDsPhase@3495c204 - Handle: urn:uuid:f9425351-5e7a-####-####-########9ec, Current Phase: RelocateStoredVmByDatastoreActivity$MigratePoweredOnVmAcrossRpAndDsPhase | requestId=UI-15f113ff-2b02-####-####-########f7d-5O-15b4de70-7da4-####-####-########728,request=PUT https://vcd_url/api/vApp/vm-b661b1a5-6e78-46ce-9952-6ff5565b1839/networkConnectionSection,requestTime=1608115217564,remoteAddress=10.33.66.160:49868,userAgent=Apache-HttpClient/4.5.12 (Java/11.0.7),accept=application/*+xml;version 31.0 vcd=3fd851b6-7683-####-####-########6cd,task=c02a2e54-01cc-####-####-########616 activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:c02a2e54-01cc-####-####-########616) activity=(com.vmware.ssdc.backend.services.impl.RelocateVmActivity,urn:uuid:25105f40-1690-####-####-########339) activity=(com.vmware.vcloud.fabric.storage.storedVm.impl.RelocateStoredVmByStorageClassActivity,urn:uuid:d29320d9-abf5-####-####-########839) activity=(com.vmware.vcloud.fabric.storage.storedVm.impl.RelocateStoredVmByDatastoreActivity,urn:uuid:f9425351-5e7a-####-####-########9ec)
java.util.concurrent.ExecutionException: com.vmware.ssdc.util.RelocateException: null
at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:136)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.vmware.ssdc.util.RelocateException: null
at com.vmware.vcloud.val.internal.impl.VC20VirtualServer.relocateVm(VC20VirtualServer.java:2397)
at com.vmware.vcloud.val.internal.impl.VC20VirtualServer.relocate(VC20VirtualServer.java:2323)
at com.vmware.vcloud.fabric.storage.storedVm.impl.RelocateStoredVmByDatastoreActivity$MigratePoweredOnVmAcrossRpAndDsPhase.baseInvoke(RelocateStoredVmByDatastoreActivity.java:274)
...
2021-02-18 08:46:49,862 | ERROR | storage-fabric-activity-pool-303 | CreateFromVCenterVmActivity | [Activity Execution] Uncaught Exception during Activity execution. Recent phase: com.vmware.vcloud.fabric.storage.storedVm.impl.CreateFromVCenterVmActivity$ImportFromVcPhase@127539ee - Handle: urn:uuid:fd4e52ef-4821-4dbf-bd73-4badb66c5964, Current Phase: CreateFromVCenterVmActivity$ImportFromVcPhase | requ
estId=UI-b71196cf-97a3-####-####-########f8d-kZ-Tn-Ms2-369d28e9-960e-####-####-########3dd,request=POST https://vcd_url/api/admin/extension/vimServer/49062a39-####-####-########ed0/importVm
IntoExistingVApp,requestTime=1613638003533,remoteAddress=10.58.196.19:44118,userAgent=Apache-HttpClient/4.5.12 (Java/11.0.7),accept=application/*+xml;version 31.0 vcd=8fd967d2-6352-####-####-########595,ta
sk=693c4f3b-e400-####-####-########fbc activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:693c4f3b-e400-####-####-########fbc) activity=(com.vmware.vcloud.vdc.impl.ImportVmIntoExistingVAppActivity,urn:uuid:9e3fab74-d819-####-####-########b28) activity=(com.vmware.vcloud.vdc.impl.CreateStoredVmActivity,urn:uuid:fd4e52ef-4821-####-####-########964) activity=(com.vmware.vcloud.fa
bric.storage.storedVm.impl.CreateFromVCenterVmActivity,urn:uuid:fd4e52ef-4821-####-####-########964)
java.util.concurrent.ExecutionException: com.vmware.vcloud.val.exceptions.VimRemoteException: (vim.fault.MethodDisabled) {
faultCause = null,
faultMessage = null,
source = H4-29208eeb-a238-####-####-########ee3
}
at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:136)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.vmware.vcloud.val.exceptions.VimRemoteException: (vim.fault.MethodDisabled) {
faultCause = null,
faultMessage = null,
source = H4-29208eeb-a238-####-####-########ee3
}
at com.vmware.vcloud.val.internal.impl.VC20VirtualServer.relocateVm(VC20VirtualServer.java:2261)
at com.vmware.vcloud.val.internal.impl.VC20VirtualServer.relocate(VC20VirtualServer.java:2188)
...
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.