Storage vMotion fails for a virtual machine with an error "Disk size reported by storage (VASA provider) is inconsistent"

Products

VMware vSphere ESXi

Issue/Introduction

You attempt to migrate a virtual machine from one datastore to another, or try to migrate the VM between 2 ESXi hosts which do not share a common datastore
The migration failes with an error message:

Failed waiting for data. Error 195887167. Connection closed by remote host, possibly due to timeout.
Error encountered while unstunning the virtual machine for Storage vMotion.

When reviewing the vmkernel.log under /var/run/log/ of the ESXi host where the VM is running, you find the following error stack:

vmkernel.log
..
2024-03-26T12:45:29.033Z cpu31:5694759)Hbr: 3561: Migration start received (worldID=5694760) (migrateType=1) (event=0) (isSource=1) (sharedConfig=0)
2024-03-26T12:45:29.033Z cpu37:5699330)MigrateNet: 1751: 3333737544153308437 S: Successfully bound connection to vmknic vmk1 - 'xxx.xxx.xxx.xxx'
2024-03-26T12:45:29.034Z cpu51:2099215)MigrateNet: vm 2099215: 3263: Accepted connection from <::ffff:xxx.xxx.xxx.xxx>
2024-03-26T12:45:30.643Z cpu35:5694767)WARNING: SVM: 5114: scsi0:2 Disk size reported by storage (VASA provider) of 1084479242240 bytes is inconsistent with the size reported by vmx (912680550400). Failing migration.
2024-03-26T12:45:30.669Z cpu54:5694939)WARNING: Migrate: 6145: 3333737544153308437 S: Migration considered a failure by the VMX. It is most likely a timeout, but check the VMX log for the true error.
2024-03-26T12:45:30.669Z cpu54:5694939)WARNING: Migrate: 282: 3333737544153308437 S: Failed: Migration determined a failure by the VMX (0xbad0092) @0x418034fcd941
2024-03-26T12:45:30.669Z cpu54:5694939)Hbr: 3655: Migration end received (worldID=5694760) (migrateType=1) (eve..
..

Environment

vCenter Server 7.0

vCenter Server 8.0

ESXi 7.0

ESXi 8.0

Cause

This error occurs when the disk size in the descriptor file of a virtual disk is different from the size of the disk reported by the VASA provider.

Resolution

Start with finding out the actual size of the disk. Going with the example above, the provider reports it as 1084479242240 bytes.

1084479242240 bytes / (1024*1024*1024) = 1010 GByte

So the actual size of the virtual disk is 1010 GByte.
Next, review the VMDK descriptor file and look for the line starting with RW:

# Disk DescriptorFile
version=3
encoding="UTF-8"
CID=86022472
parentCID=ffffffff
createType="vmfs"

# Extent description
RW 1782579200 VMFS "vmname-flat.vmdk"

# Change Tracking File
changeTrackPath="vmname-ctk.vmdk"

# The Disk Data Base
#DDB

ddb.adapterType = "lsilogic"
ddb.deletable = "true"
ddb.geometry.cylinders = "110960"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "3f8978d734b845d9fbfaa51786022472"
ddb.toolsInstallType = "1"
ddb.toolsVersion = "11333"
ddb.uuid = "60 00 C2 9e 66 a9 f3 30-00 b9 f7 36 2a 88 23 0d"
ddb.virtualHWVersion = "9"

The value in this line is the number of sectors in the virtual disk.
The Sector size is 512 bytes, therefore you need to multiple the value in the descriptor by 512 to get the actual size value in bytes:

1782579200 sectors * 512 bytes = 912680550400 bytes

Again divide by 1024^3 to get the size in GBytes:

912680550400 bytes / (1024 * 1024 * 1024) = 850 GBytes

So according to the descriptor file, this disk is 850 GBytes, but the VASA provider actually reports it to be 1010 GByte, hence the error occurs.

To fix this issue, you need to update the VMDK descriptor file with the actual size of the virtual disk as it is being reported by the VASA provider. For this, follow the steps below:

Open the vSphere Client and login
Find the problematic virtual machine, right-click it and select "Edit Settings" to open the VM settings wizard
Select the disk with the invalid size. If the VM has a larger number of disks attached, expand the "Hard Disks section" and identify the disk.
Replace the current disk size (going by the example above this would be 850 GB) against the size reported by the VASA provider according to the vmware.log (for our example enter 1010 instead of the 850), then click on [ OK ] to close the wizard:

You should now be able to successfully migrate the virtual machine.

Additional Information

Similar issues might occur when there is a corruption on the underlying storage. In such cases, you will not be able to solve the problem with the steps above.

Should you experience this, please restore the virtual disk or the whole virtual machine from its last know good backup, or recreate it from scratch (after making sure that the corruption has been fixed), or involve one of our partners to assist with the recovery of the data.