Manually removing a stale virtual machine from vCenter Server

Products

• VMware vCenter Server

Issue/Introduction

This article provides instructions for removing orphaned virtual machines and/or templates from the vCenter Server database. This procedure is used when the objects cannot be removed via the vSphere Client.

Symptoms:

• A virtual machine appears as "orphaned" or "inaccessible" in the vSphere Client
• The "Remove from Inventory" or "Delete from Disk" option is grayed out and cannot be selected
• The virtual machine is not listed when the command vim-cmd vmsvc/getallvms is run on the ESXi host CLI
• Attempts to delete a VM template may result in the error: "The object 'vim.VirtualMachine:vm-####' has already been deleted or has not been completely created"
• The "Remove from Inventory" or "Delete from Disk" option is grayed out and cannot be selected

 

 

Environment

VMware vCenter Server 6.5.x
VMware vCenter Server 6.7.x
VMware vCenter Server 7.x
VMware vCenter Server 8.x
VMware vCenter Server 9.0.x

Cause

Metadata inconsistency occurs when a virtual machine or datastore is removed directly from an ESXi host or storage layer while vCenter Server is unaware of the change; leaving residual entries in the vCenter Server database.

Resolution

Note: Create an offline snapshot of the vCenter Server Appliance before proceeding.

If the vCenter Server is part of a Linked Mode replication group, OFFLINE backups must be created for every member as detailed in KB Snapshot Best practices for vCenter Server Virtual Machines

Method 1: Dummy Folder Workaround

1. In the vSphere Client, change into the "VMs and Folders" view
2. Create a new Virtual Machine folder
3. Drag and drop the orphaned virtual machine into this folder
4. Delete the folder

Method 2: Manual Database Removal (Single VM Object) 

1. SSH to the vCenter Server as root
2. Stop the vpxd service:

   service-control --stop vmware-vpxd

3. Connect to the database:

   /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

4. Identify the VMID by searching for the VM name (where <vm_name> is the actual name of the orphaned VM):

   select id, name from vpx_entity where name like '%<vm_name>%';

     Example Output:

        VCDB=# select * from vpx_entity where name like '%vcsa%';
      id      |   name                | type_id  | parent_id
     --------+---------------------+------------+------------
      0000 | vcsa(orphaned) |    #        |    #

5. Execute the following delete statements in order, replacing #### with the numeric id found previously:

        delete from VPX_COMPUTE_RESOURCE_DAS_VM where VM_ID=<VM_ID>;
   
        delete from VPX_COMPUTE_RESOURCE_DRS_VM where VM_ID=<VM_ID>;
   
        delete from VPX_COMPUTE_RESOURCE_ORC_VM where VM_ID=<VM_ID>;
   
        delete from VPX_VM_SGXINFO where VM_ID=<VM_ID>;
   
        delete from VPX_GUEST_DISK where VM_ID=<VM_ID>;
   
        delete from VPX_VM_VIRTUAL_DEVICE where ID=<VM_ID>;
   
        delete from VPX_VM_DS_SPACE where VM_ID=<VM_ID>;
   
        delete from VPX_NON_ORM_VM_CONFIG_INFO where ID=<VM_ID>;
   
        delete from VPX_NORM_VM_FLE_FILE_INFO where VM_ID=<VM_ID>;
   
        delete from VPX_VDEVICE_BACKING_REL where VM_ID=<VM_ID>;
   
        delete from VPX_VIRTUAL_DISK_IOFILTERS where VM_ID=<VM_ID>;
   
        delete from VPX_VM_STATIC_OVERHEAD_MAP where VM_ID=<VM_ID>;
   
        delete from VPX_VM_TEXT where VM_ID=<VM_ID>;
   
        delete from VPX_VM where ID=<VM_ID>;
   
        delete from VPX_ENTITY where ID=<VM_ID>;
   
        delete from VPX_DVPORT where connectee='<VM_Name>';

6. Start the vpxd service:

   service-control --start vmware-vpxd

Additional Information

Review Snapshot Best practices for vCenter Server Virtual Machines before performing database maintenance.