"Some of the disks of the virtual machine failed to load" after an unsuccessful backup operation

Products

• VMware vSphere ESXi

Issue/Introduction

• The virtual machines have the following warning:
  Some of the disks of the virtual machine failed to load.


• The backup operation fails with the below error: 
  Failed to fetch the virtual machine disks.
  
  
• The virtual machine has an ISO attached.
  
  This can be validated as follows:
  • Right-click the desired virtual machine in the inventory and select Edit Settings.
  • In the Virtual Machine Properties window, navigate to the CD/DVD drive (often listed as CD/DVD Drive 1).
  • Check the details in the right pane. It should list "Datastore ISO File" and the full path to the .iso file if one is attached.
  • Ensure the "Connected" or "Connect At Power On" checkbox is selected for the guest OS to access it.
    
    
• Virtual machine disk file size shows 0. This can be validated following the same steps as the above step. 

Environment

VMware vCenter Server 8.x
VMware vSphere ESXi 8.x

Cause

While virtual machines consist of multiple files, the ESXi host service (hostd) processes it as a single object. Should any of the virtual machine's child items become inaccessible, the virtual machine will transition into an invalid load state.

For example, if an ISO has been attached to a virtual machine and that ISO was moved away from the datastore; or was deleted without removing it from the virtual machine beforehand, the virtual machine will experience the issues described.

Furthermore, when the datastore that the virtual machine is located on becomes inaccessible, hostd will not refresh the disk. Default entries with 0 capacity will be generated for those virtual disks that were not yet cached. In this situation, hostd will not refresh the snapshot list for the virtual machine. As a result, snapshot creation tasks will fail. The snapshot itself is actually getting created, but the system fails to find the new UUID in the list, resulting in the task failing.

Resolution

Ideally in this scenario, if we are keeping the ISO attached to the virtual machine, the datastore containing the ISO should be available to the ESXI.

One of the following action can be performed to mitigate this warning. 

• Remove the .ISO file from the virtual machine.
• We can also try to vMotion the virtual machine to an ESXi host where the datastore is accessible. 
• Mount the datastore to the ESXi host where the virtual machine is present at the time of the issue .
• Resolve if there is any underlying issue with storage.

After performing the above action, and confirming the virtual machine is powered off, please follow the below steps to re-register the virtual machine.

• Launch the vSphere web client and log in with the appropriate credentials.
• Navigate to the desired ESXi host.
• Right click on the virtual machine and select unregister.
• Then select the configure tab on datastore. 
• Under datastores, right-click the appropriate datastore and select browse files.
• Locate the virtual machine's directory and find the corresponding .vmx file.
• Right-click the .vmx file and select register virtual machine.
• Follow the registration wizard to complete the process.

The warning should be gone now and backup should work as expected.