vim.fault.GenericVmConfigFault when deleting a virtual machine or during power operations or snapshot management in vSphere
search cancel

vim.fault.GenericVmConfigFault when deleting a virtual machine or during power operations or snapshot management in vSphere

book

Article ID: 374299

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Virtual machine operations such as reboot, reset, consolidation, or deletion fail with a system error. This issue commonly occurs during disk management, power state transitions, or Site Recovery Manager (SRM) workflows.

Symptoms

  • Error when rebooting or resetting a VM:
    Method vim.VirtualMachine.rebootGuest threw undeclared fault of type vim.fault.GenericVmConfigFault

     

  • Error when deleting or consolidating snapshots in vSphere:

    [root@hostname:~] vim-cmd vmsvc/snapshot.removeall 3
Remove All Snapshots:
Remove all snapshots failed: (vmodl.fault.SystemError) {
   faultCause = (vmodl.MethodFault) null,
   faultMessage = <unset>,
   reason = "Fault cause: vim.fault.GenericVmConfigFault"
   msg = "A general system error occurred: Fault cause: vim.fault.GenericVmConfigFault.
  • SRM Reprotect fails with: Unable to reverse replication for the virtual machine... A general system error occurred: Fault cause: vim.fault.GenericVmConfigFault.

  • The var/run/log/hostd.log file reports "An error occurred while deleting a snapshot: One of the disks in this virtual machine is already in use by a virtual machine or by a snapshot".

    YYYY-MM-DDTHH:MM:SSZ verbose hostd[21##69] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:################-################/########-####-####-####-############/VM_NAME.vmx opID=m6###xyt-417##903-auto-ou5qo-h5:739##492-f1-12-772b user=vpxuser:####\#######] Removeallsnapshots received. Consolidate: true
YYYY-MM-DDTHH:MM:SSZ info hostd[2104769] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:################-################/########-####-####-####-############/VM_NAME.vmx opID=m6##6xyt-417###03-auto-ou5qo-h5:7####492-f1-12-772b user=vpxuser:####\#######] State Transition (VM_STATE_ON -> VM_STATE_REMOVEALL_SNAPSHOT)
YYYY-MM-DDTHH:MM:SSZ verbose hostd[210##2] [Originator@6876 sub=Vigor.Vmsvc.vm:/vsan:################-################/########-####-####-####-############/VM_NAME.vmx opID=m####xyt-41##3861-auto-on7pi-h5:7###4520-ad-01-01-01-01-6a-59ae] Delete snapshot 2##6 and its children message: An error occurred while deleting a snapshot: One of the disks in this virtual machine is already in use by a virtual machine or by a snapshot.
-->
YYYY-MM-DDTHH:MM:SSZ warning hostd[21##69] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:################-################/########-####-####-####-############/VM_NAME.vmx opID=m####6xyt-41####03-auto-ou5qo-h5:7####492-f1-12-772b user=vpxuser:####\#######] Failed to remove snapshot(s): N3Vim5Fault20GenericVmConfigFault9ExceptionE(Fault cause: vim.fault.GenericVmConfigFault
--> )

  • The /vmfs/volumes/datastore_UUID/VM_NAME/vmware.log file reports the following error. 

    YYYY-MM-DDTHH:MM:SSZ In(05) vmx m6836xyt-41###903-auto-ou5qo-h5:7##25492-f1-12-772b SNAPSHOT: SnapshotDiskTreeAddFromSnapshot: Trying to add snapshot VM_NAME-Snapshot2##7.vmsn to
 disk /vmfs/volumes/vsan:################-################/########-####-####-####-############/VM_NAME.vmdk which already has snapshot VM_NAME-Snapshot2##6.vmsn.
YYYY-MM-DDTHH:MM:SSZ In(05) vmx m6836xyt-41###903-auto-ou5qo-h5:7##25492-f1-12-772b SNAPSHOT: SnapshotGenerateDeleteDisks Failed to fetch disk tree: One of the disks in this virtual machine is already in use by a virtual machine or by a snapshot (21)
YYYY-MM-DDTHH:MM:SSZ In(05) vmx m6836xyt-4###7903-auto-ou5qo-h5:7###5492-f1-12-772b SNAPSHOT: SnapshotDeleteNode failed: One of the disks in this virtual machine is already in use by avirtual machine or by a snapshot (21)
YYYY-MM-DDTHH:MM:SSZ In(05) vmx m6836xyt-41717903-auto-ou5qo-h5:7###5492-f1-12-772b SNAPSHOT: Snapshot_Delete failed: One of the disks in this virtual machine is already in use by a virtual machine or by a snapshot (21)

  • Example of Incorrect .vmsd File Hierarchy:

    less /vmfs/volumes/<datastore_uuid>/VM_Name/vm_name.vmsd

    .encoding = "###-#"
snapshot.lastUID = "7"
snapshot.current = "7"
snapshot0.uid = "5"
snapshot0.filename = "vm_name-Snapshot5.vmsn"
snapshot0.displayName = "VM Snapshot DATE, TIME"
snapshot0.createTimeHigh = "408330"
snapshot0.createTimeLow = "-206508375"
snapshot0.numDisks = "3"
snapshot0.disk0.fileName = "vm_name.vmdk"
snapshot0.disk0.node = "scsi0:0"
snapshot0.disk1.fileName = "vm_name_1.vmdk"
snapshot0.disk1.node = "scsi0:1"
snapshot0.disk2.fileName = "vm_name_2.vmdk"
snapshot0.disk2.node = "scsi0:2"
snapshot.numSnapshots = "3"
snapshot1.uid = "6"
snapshot1.filename = "vm_name-Snapshot6.vmsn"
snapshot1.parent = "5"
snapshot1.displayName = "VM Snapshot DATE, TIME"
snapshot1.createTimeHigh = "408330"
snapshot1.createTimeLow = "-200076526"
snapshot1.numDisks = "3"
snapshot1.disk0.fileName = "vm_name.vmdk"      <<< snapshot1.disk0.fileName should point to vm_name-000001.vmdk instead of vm_name.vmdk. The same applies to the other disks. >>>
snapshot1.disk0.node = "scsi0:0"
snapshot1.disk1.fileName = "vm_name_1.vmdk"
snapshot1.disk1.node = "scsi0:1"
snapshot1.disk2.fileName = "vm_name_2.vmdk"
snapshot1.disk2.node = "scsi0:2"
snapshot2.uid = "7"
snapshot2.filename = "vm_name-Snapshot7.vmsn"
snapshot2.parent = "6"
snapshot2.displayName = "VM Snapshot DATE, TIME"
snapshot2.createTimeHigh = "408330"
snapshot2.createTimeLow = "-191878359"
snapshot2.numDisks = "3"
snapshot2.disk0.fileName = "vm_name-000001.vmdk"
snapshot2.disk0.node = "scsi0:0"
snapshot2.disk1.fileName = "vm_name_1-000001.vmdk"
snapshot2.disk1.node = "scsi0:1"
snapshot2.disk2.fileName = "vm_name_2-000001.vmdk"
snapshot2.disk2.node = "scsi0:2"

     

Environment

  • VMware vSphere ESXi 7.0.x / 8.0.x

  • VMware Live Site Recovery 9.0

  • VMware vSphere Replication 8.x

Cause

Corruption or invalid hierarchy in the .vmsd snapshot metadata file or .vmsn state files prevents vSphere from correctly fetching the disk tree or completing power state transitions.

Resolution

  1. Using the vSphere Client, shut down the guest operating system and power the virtual machine off.

  2. Locate the virtual machine folder where the .vmx configuration file of the VM resides.

  3. Open an SSH session to the host where the VM is currently registered.

    (Note: The file /etc/vmware/hostd/vmInventory.xml lists all VMs the host currently holds with their configuration file paths).

  4. Alternatively, use the following command to list all virtual machines on the host:

    esxcli vm process list

    Or, search for the specific .vmx file:

    find /vmfs/volumes -iname <vm_name>.vmx

  5. Note the path to the .vmx file and unregister the VM from the vCenter inventory. To do this, right-click the VM in the vSphere Client and select Unregister from Inventory. Do not select Delete from Disk.

  6. Return to the SSH session with the ESXi host, and navigate to the VM folder noted in Step 4:

    cd /vmfs/volumes/<datastore_name>/<vm_name>

  7. Create a temporary folder within this VM folder:

    mkdir vm_snapshot_temp

  8. Move all files with the extensions .vmsn (snapshot state files) and .vmsd (snapshot directory file) from the VM folder into this new folder:

    mv *.vmsn ./vm_snapshot_temp/
    mv *.vmsd ./vm_snapshot_temp/

  9. Return to the vSphere Client and use the datastore browser to locate the .vmx file of the virtual machine. Select the file and click Register VM to re-register the virtual machine with the vCenter Server.  

  10. Acknowledge the ‘VM consolidation needed’ status and wait for the consolidation process to complete.

  11. Once the consolidation process finishes, verify that there are no more snapshots in the virtual machine folder, then power the virtual machine back on.

  12. For SRM environments, re-run the Reprotect task.

Note: The consolidation process may finish without an error but confirm by checking the attached disks of the VM afterwards, and check the folder. You may see that the VM is still running on snapshots. If that is the case:

1. Attempt by taking a temporary snapshot, and then deleting all snapshots.

2. If that does not still consolidate snapshots, you can clone the VM to another datastore to consolidate snapshots by cloning. To perform it:

                             a. Rename the VM. This is required for consolidating by cloning to happen
                             b. svmotion the VM to another datastore.
                             c. The VM with its files will be renamed to the new name, consolidating the disks during the process.

After moving the .vmsn and .vmsd files and re-registering the virtual machine, If removing all snapshots - click Remove All Snapshots after registration. If deleting the VM entirely, select Delete from Disk.

  1.  

Additional Information

Powered by Wolken Software