Error: "Internal Server Error" VM power on failure due to stale Resource Pool entry

Products

• VMware Cloud Director

Issue/Introduction

• Powering on VM's in Cloud Director fail with the following error "Internal Server Error"
• VM's are in an inconsistent state in the Cloud Director UI  
• In the /opt/vmware/vcloud-director/logs/vcloud-container-debug.log you will see entries similar to:
  
  
  2026-04-20 10:41:08,207 | ERROR    | Backend-activity-pool-10  | GenericVirtualResourcePool     | vrp <resource-pool-name> (########-####-####-####-############):########-####-####-####-############ - Failed to retrieve info of resource pool [vcId=########-####-####-####-############, moref=resgroup-######]. | requestId=########-####-####-####-############,request=PUT https://example.vcloud.com/api/vApp/vm-########-####-####-####-############/networkConnectionSection/,requestTime=1776681660651,remoteAddress=###.##.###.##:55656,userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ...,accept=application/*+xml;version 40.0.0-alpha vcd=########-####-####-####-############,task=########-####-####-####-############ activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:########-####-####-####-############) activity=(com.vmware.ssdc.backend.services.impl.RelocateVmActivity,urn:uuid:########-####-####-####-############)
  ...
  2026-04-20 10:41:08,208 | ERROR    | Backend-activity-pool-10  | RelocateVmActivity             | [Activity Execution] Uncaught Exception during Activity execution. Recent phase: com.vmware.ssdc.backend.services.impl.RelocateVmActivity$ConfigureSourceHubIfNeeded@######## - Handle: urn:uuid:########-####-####-####-############, Current Phase: RelocateVmActivity$ConfigureSourceHubIfNeeded | requestId=########-####-####-####-############,request=PUT https://vcloud.example.com/api/vApp/vm-########-####-####-####-############/networkConnectionSection/,requestTime=1776681660651,remoteAddress=###.##.###.##:55656,userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ...,accept=application/*+xml;version 40.0.0-alpha vcd=########-####-####-####-############,task=########-####-####-####-############ activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:########-####-####-####-############) activity=(com.vmware.ssdc.backend.services.impl.RelocateVmActivity,urn:uuid:########-####-####-####-############)
java.util.concurrent.ExecutionException: com.vmware.vcloud.fabric.compute.ResourcePoolNotFoundException
        at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:136)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.vmware.vcloud.fabric.compute.ResourcePoolNotFoundException
        at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.getResourcePoolInfo(GenericVirtualResourcePool.java:1612)
        at com.vmware.vcloud.fabric.compute.vrp.AllocationPoolVirtualResourcePool.computeResourcePoolConfig(AllocationPoolVirtualResourcePool.java:282)
        at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.reconfigureOtherResourcePoolsForNonFlexVdc(GenericVirtualResourcePool.java:1424)
        at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool$ReconfigOtherResourcePoolRunnableContext.run(GenericVirtualResourcePool.java:5160)
        at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool$ReconfigOtherResourcePoolRunnableContext.run(GenericVirtualResourcePool.java:5141)


Environment

VMware Cloud Director 10.6.X
VMware Cloud Director 10.5.X

Cause

This issue occurs due to the OVDC pointing to a resource pool that no longer exists in vCenter.

Resolution

To resolve this issue perform the following steps:

1. SSH to one of the Cloud Director cells:
   
   ssh root@<vcd_ip_address>


2. Run the following command to detect resource pool mismatches between Cloud Director and vCenter:
   
   /opt/vmware/vcloud-director/bin/cell-management-tool detect-rp-mismatches --detect-mismatch-with-vc --vdcs <Organization VDC Name>


3. If resource pool mismatches are detected for the affected VDC please run the following command to fix the mismatches:
   
   /opt/vmware/vcloud-director/bin/cell-management-tool detect-rp-mismatches --fix-mismatch-with-vc --vdcs <Organization VDC Name>
   
   

 

If the steps above do not resolve the issue, contact Broadcom Technical Support and note this Knowledge Article ID (438927) in the problem description as the resolution requires a database edit. For more information, see How to Submit a Support Request