Power on failed with error "Caused by: com.vmware.vcloud.fabric.compute.ResourcePoolNotFoundException"

Products

• VMware Cloud Director

Issue/Introduction

In VCD Database VM resource pool details are not matching with VCenter Database.

vcloud-container-info.log

xxxx-xx-xx xx:xx:xx,xxx | ERROR    | Backend-activity-pool-###7 | GenericVirtualResourcePool     | vrp ORG_NAME (####-####-####-####):####-####-####-#### - Failed to retrieve info of resource pool [vcId=#####-####-###-####-#######, moref=resgroup-#####].

Environment

VCD 10.x

Cause

Any operation we perform on the VM, we get below UI error due to DB mismatch:

com.vmware.ssdc.util.LMException: Internal Server Error
	at com.vmware.ssdc.util.LMException.wrap(LMException.java:135)
	at com.vmware.ssdc.library.ExceptionFactory.createFromMultiple(ExceptionFactory.java:32)
	at com.vmware.ssdc.backend.DeployVAppVmSubsetActivity.checkFuturesForException(DeployVAppVmSubsetActivity.java:529)
	at com.vmware.ssdc.backend.DeployVAppVmSubsetActivity$DeployNetworksPhase.invoke(DeployVAppVmSubsetActivity.java:210)
	at com.vmware.vcloud.activity.executors.ActivityRunner.runPhase(ActivityRunner.java:175)
	at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:112)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.vmware.vcloud.api.presentation.service.InternalServerErrorException: Internal Server Error
	at com.vmware.vcloud.common.future.FutureUtil.convertExecutionException(FutureUtil.java:264)
	at com.vmware.vcloud.common.future.FutureUtil.checkCompletedFuture(FutureUtil.java:203)
	at com.vmware.ssdc.backend.DeployVAppVmSubsetActivity.checkFuturesForException(DeployVAppVmSubsetActivity.java:522)
	... 8 more
Caused by: com.vmware.vcloud.fabric.compute.ResourcePoolNotFoundException
	at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.getResourcePoolInfo(GenericVirtualResourcePool.java:1612)
	at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.computeResourcePoolConfig(GenericVirtualResourcePool.java:1993)
	at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.computeAndAdjustDelta(GenericVirtualResourcePool.java:4458)
	at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.deployAdmittedVms(GenericVirtualResourcePool.java:3397)
	at com.vmware.vcloud.fabric.compute.vrp.GenericVirtualResourcePool.deployVms(GenericVirtualResourcePool.java:496)
	at com.vmware.vcloud.fabric.compute.impl.ComputeFabricImpl.deployVms(ComputeFabricImpl.java:356)
	at jdk.internal.reflect.GeneratedMethodAccessor21318.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:234)
	at com.sun.proxy.$Proxy273.deployVms(Unknown Source)
	at com.vmware.ssdc.backend.DeployVmOnFabricsActivity$DeployVMsComputeFabricPhase.invoke(DeployVmOnFabricsActivity.java:156)
	... 7 more

Resolution

Workaround:

1.  Make a note of resource pool from log moref=resgroup-12345
2. Switch to Vcenter and highlight the resource pool where the VM is residing , URL like below:

   1. https://vcsa.example.local/mob/?moid=resgroup-56789

3. Comparing the log snippet value and actual value in Vcenter is not matching 
4. Move the VM to actual resource in Vcenter level 
   1. Note: Destination resource pool should be part of Organization 
5. Then re-connect Vcenter from Vcloud and then try powering ON.