Stranded Items folder in vCenter containing one or more VMs which are not present in VMware Cloud Director

Products

• VMware Cloud Director

Issue/Introduction

• Stranded items folders seen in VC inventory with names starting with "Stranded .." followed by a VMware Cloud Director(VCD) tenant name.
• Stranded items folder in VC contain VMs which do not exist in VCD.
• No reference of the VMs are present in VCD UI or database.

Environment

VMware Cloud Director 10.4.x

VMware Cloud Director 10.6.x

Cause

The issue with the VMs in the stranded folder in vCenter occurs when VCD deletes a VM, but that VM is actually not deleted from VC/ESXi.
The VM is then discovered again in Organization VDC(OVDC) RP, but VCD does not know whom it belongs and puts it into stranded VM folder in VC.
In terms of multiple folders, VCD will only attempt to create the stranded items folder only if it cannot find it. The folder duplication can happen because the already existing stranded folder has a different name from the one VCD would use which can be a consequence of temporary issues syncing with VC during the process or VC-ESXi connectivity issues.

Resolution

To verify if the VM(s) is/are seen in VCD, please follow the steps as below:

1. Search for the VM with its name from the provider or the tenant portal.
2. To verify if the VM(s) is/are present in the database:
   1. In VC, click through each VM in the stranded items folder and note the moref of the VM from the URL.
      1. Open a web browser and navigate to https://<vcenter_server_ip_address_or_fqdn>/ui
      2. Authenticate with your vCenter credentials.
      3. In the Navigate to the "Hosts and Clusters" or "VMs and Templates" view,then the Stranded items folder and click on one VM at a time. 
      4. The URL will change to the format as below:
         https://<vcenter_server_ip_address_or_fqdn>/ui/app/vm;nav=v/urn:vmomi:VirtualMachine:vm-#####:########-####-####-####-############/summary
      5. Note the moref value from the URL. For example from the above URL, the moref is: vm-#####
         
         
   2. Log in directly or by using an SSH client to the primary appliance console as root.Connect to the vcloud database with:
      
      sudo -u postgres psql vcloud


   3. Using the moref noted from the VC, populate the moref value in the following script and run it to identify for any entries in VM related tables:
      
      select 
vm.name as vmname,
vm.id as vmid,
vm.moref as vmmoref,
vmc.name as vapp,
vmc.org_vdc_id as ovdcid,
cvm.id as cvmid,
cvm.deployment_status as cvmstatus,
dvm.instance_id as dvmid,
vinv.is_deleted as vcvmstats
from vm 
left join vapp_vm vvm on vvm.svm_id = vm.id
left join vm_container vmc on vmc.sg_id = vvm.vapp_id
left join computevm cvm on cvm.id = vvm.cvm_id
left join deployed_vm dvm on dvm.vm_id = vm.id
left join vm_inv vinv on vinv.moref =vm.moref
where vm.moref = '<vm moref from Step 2.1>';
      
      
   4. There won't be any entries present in any of the tables other than vm_inv(vcenter inventory table for VMs) in normal circumstances. If there are any stale records present, please refer to KB 371530 
3. Once the VMs are confirmed as non-existing in VCD, the objects can now be manually removed from VC, please follow the steps as per https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/remove-virtual-machines-from-vcenter-server.html