Orphaned Shadow VMs managed by cloud director is Displaying "Consolidation Needed" Status in vCenter Server

Products

• VMware Cloud Director

Issue/Introduction

• VMware Cloud Director (VCD) template shadow VMs are displaying a "Consolidation Needed" status in vCenter Server.
• This invalid state can prevent backup jobs from processing the environment successfully.
• The /opt/vmware/vcloud-director/logs/vcloud-container-debug.log on the VMware Cloud Director Cells shows below logs:
  
  WARN | pc-activity-pool-9 | StoredVirtualMachineListenerImpl | shadow VM moref=[vcId=########-####-####-########, moref=vm-######] no longer valid for datastore-moref=datastore-######, oldVmPathName=[nfs######-####] shadow-VM-Name/shadow-VM-Name.vmx, newVmPathName=[nfs######-####] shadow-VM-Name/shadow-VM-Name.vmx, setting dummy datastore ref to datastore-1########-####-####-######## | activity=(com.vmware.vcloud.vimproxy.internal.impl.PCEventProcessingActivity,urn:uuid:########-####-####-########)

WARN | pc-activity-pool-5 | StoredVirtualMachineListenerImpl | shadow VM moref=[vcId=########-####-####-########, moref=vm-###] no longer valid for datastore-moref=datastore-##, oldVmPathName=[shared-disk-1] shadow-VM/shadow-VM-Name.vmx, newVmPathName=[shared-disk] shadow-VM/shadow-VM-Name.vmx, setting dummy datastore ref to datastore-########-####-####-######## | activity=(com.vmware.vcloud.vimproxy.internal.impl.PCEventProcessingActivity,urn:uuid:########-####-####-########)

Environment

VMware Cloud Director 10.x

Cause

Shadow VMs are datastore-bound caching objects that VMware Cloud Director dynamically creates to support fast-provisioned linked clones on a specific local datastore.

This issue occurs when shadow VMs are moved across datastore boundaries outside of VMware Cloud Director's management, such as through automated Storage DRS (SDRS) placement or manual Storage vMotion operations performed directly in the vSphere layer. Moving these VMs breaks their expected datastore association, resulting in orphaned shadow VMs that may appear in an invalid state, including "Consolidation Needed."

Resolution

Run the attached find_shadow_dependencies.py script using the command below to locate orphaned shadow VMs and display any dependent tenant VMs:

python find_shadow_dependencies_1.py -v "vCenter-FQDN" -u "administrator@domain" -p "Password" -s "shadow-VM"


If dependent VMs are found, stale entries must be cleared from the Cloud Director database. To resolve this issue, please open a Support Request with Broadcom Technical Support and note this Article ID (446435) in the problem description. For more information, see Creating and managing Broadcom support cases.

Additional Information

To prevent future occurrences of database corruption and consolidation warnings on template caches, configure Storage DRS (SDRS) Exclusions within vCenter Server to ignore any virtual machine names matching the prefix shadow-*. This ensures vSphere will never automatically move VCD template backings without Cloud Director's awareness.