Error: "Inventory timed out waiting for enable relocateVM_Task for VM..." when failing over vApp Replications

Products

• VMware Cloud Director

Issue/Introduction

• When failing over vApp replications in VMware Cloud Director Availability (VCDA), the failover times out for some of the VM replications and you see errors similar to:
  
  Unexpected VMware Cloud Director error. [ UI-########-####-####-####-########cb3e-r1661-38-JP-3K8-########-####-####-####-########c3b0 ] Inventory timed out waiting for enable relocateVM_Task for VM [vcId=########-####-####-####-########8cdf, moref=vm-###]
  
  
• In /opt/vmware/vcloud-director/logs/vcloud-container-debug.log on the VMware Cloud Director (VCD) cell that performs a post failover reconfigure task on the recovered VM, you see entries similar to:
  
  2025-06-19 09:32:15,193 | ERROR    | Backend-activity-pool-98330 | RelocateVmActivity             | An exception occurred while relocating to destination hub moref://########-####-####-####-########28cdf/ResourcePool#resgroup-### attempting to reassign vm to source hub urn:uuid:########-####-####-####-########cd24, and undo destination hub moref://########-####-####-####-########8cdf/ResourcePool#resgroup-### reconfiguration if one was performed | requestId=UI-########-####-####-####-########cb3e-r1661-38-JP-3K8-########-####-####-####-########c3b0,request=PUT https://<VCD FQDN>/api/vApp/vm-########-####-####-####-########f664/networkConnectionSection,requestTime=1750325230190,remoteAddress=##.##.##.##:###,userAgent=Apache-HttpClient/4.5.14 (Java/17.0.12),accept=application/*+xml;version 38.0 vcd=########-####-####-####-########9e78,task=########-####-####-####-########55df activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:########-####-####-####-########55df) activity=(com.vmware.ssdc.backend.services.impl.RelocateVmActivity,urn:uuid:########-####-####-####-########28f6)

java.util.concurrent.TimeoutException: Inventory timed out waiting for enable relocateVM_Task for VM [vcId=########-####-####-####-########8cdf, moref=vm-#####]
        at com.vmware.ssdc.backend.services.impl.RelocateVmActivity$RelocateVmAcrossRpAndDsPhase.invoke(RelocateVmActivity.java:838)
        at com.vmware.vcloud.activity.executors.ActivityRunner.runPhase(ActivityRunner.java:175)
        ...
  
  
• This issue can occur for vApp replications with a large number of instances per VM replication and consolidation enabled.

Environment

VMware Cloud Director Availability 4.7.x

Cause

This issue occurs when the consolidation of the VM replication instances take a long time to complete, causing the task to enable the relocate VM method for the recovered VM to stall and time out in Cloud Director after 5 minutes.

Resolution

This is a known issue affecting VMware Cloud Director Availability.

Subscribe to this knowledge article to get updates on this issue.

Workaround

Performing consolidation of instances can have a potential large increase in the Recovery Time Objective (RTO) of a failover depending on the following criteria:

• The number of disks per recovered VM.
• The size of each of the VM's disks.
• The data change rate of the VM.
• The number of instances retained for the VM.

To work around this issue, leave the Instances handling after recovery setting at Default so as to not consolidate the VMs on failover. The instances can then be manually consolidated after the recover task completes.