VDI VMs are static/persistent.
VDI VMs can be reconfigured on vSphere, like Hot-Add, add an existing/new HDD. On the vSphere end works fine to Power-ON/Reboot/Reconfigure.
However, Citrix is not getting updated with the VM's latest state, and the Citrix initiates session to update but fail to receive latest update from Hypervisor end.
After Powering-Off VM from vSphere. Initiating Power-On/Reboot task from Citrix fails to power on VM, and "Reconfigure Virtual" task failures on vSphere.
VM resets to previous state and not the latest (updated) state.
For Ex on vSphere:
VM > VM edit settings > Attach existing vvol vmdk > Power ON > Reboot -------- Works fine without any issue
VM > VM edit settings > Attach existing vvol vmdk > Power ON > Power Off > Power on the VM from Citrix -------- Fails with error "VM reconfigure task failure on vSphere"
/var/run/log/hostd.log
YYYY-MM-DDTHH:mm:SS.234Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] OBJLIB-VVOLOBJ :VVolObjSetMetadata: Failed to acquire metadata from cache for VVol 'naa.68cxxxxxxxxxxxxxxxxxxxxxxxxxxxxxdd': No such file or directory
YYYY-MM-DDTHH:mm:SS.300Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] 2099590:VVOLLIB : VVolLib_UpdateVVolMetadata:7766: Successful.
YYYY-MM-DDTHH:mm:SS.301Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] OBJLIB-VVOLOBJ : VVolObjGetObjTypeVersion: Returning NS version: 1 for path: /vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk
YYYY-MM-DDTHH:mm:SS.301Z Er(163) Hostd[2099590]: [Originator@6876 sub=DiskLib opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] DISKLIB-DSCPTR: DescriptorWriteInt: Failed to update the descriptor version of '/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk': One of the parameters supplied is invalid
YYYY-MM-DDTHH:mm:SS.301Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] OBJLIB-VVOLOBJ : VVolObjGetObjTypeVersion: Returning NS version: 1 for path: /vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk
YYYY-MM-DDTHH:mm:SS.301Z Er(163) Hostd[2099590]: [Originator@6876 sub=DiskLib opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] DISKLIB-DSCPTR: DescriptorWriteInt: Failed to update the descriptor version of '/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk': One of the parameters supplied is invalid
YYYY-MM-DDTHH:mm:SS.301Z Er(163) Hostd[2099590]: [Originator@6876 sub=DiskLib opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] DISKLIB-LINK : DiskLinkNativeSnap: Failed to update snap descriptor '/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk': One of the parameters supplied is invalid
YYYY-MM-DDTHH:mm:SS.305Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] OBJLIB-VVOLOBJ : VVolObjUnlink: Deleting vVol: vvol://826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxxxxxxxxxxxxxxxxxxxxdd, asyncDelete: 0, flags: 0
We see the vvol version is still looking at version: 1 and not the updated version.
YYYY-MM-DDTHH:mm:SS.640Z In(166) Hostd[2099590]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FB01WVT174ES001.vmx opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] Storage policy for disk '/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk' specified: '<ns1:storageProfile xmlns:ns1="http://profile.policy.data.vasa.vim.vmware.com/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:StorageProfile"><ns1:constraints/><ns1:createdBy>Temporary user handle</ns1:createdBy><ns1:creationTime>YYYY-MM-DDTHH:mm:SS.382+00:00</ns1:creationTime><ns1:description>Allow the datastore to determine the best placement strategy for storage objects</ns1:description><ns1:generationId>0</ns1:generationId><ns1:lastUpdatedBy>Temporary user handle</ns1:lastUpdatedBy><ns1:lastUpdatedTime>YYYY-MM-DDTHH:mm:SS.382+00:00</ns1:lastUpdatedTime><ns1:name>VVol No Requirements Policy</ns1:name><ns1:profileId>fxxxxxxxe-storage-profile-id-5xxxxxxxxxx3</ns1:profileId></ns1:storageProfile>'
YYYY-MM-DDTHH:mm:SS.646Z In(166) Hostd[2099590]: [Originator@6876 sub=Default opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] Create Child using storage policy '<ns1:storageProfile xmlns:ns1="http://profile.policy.data.vasa.vim.vmware.com/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:StorageProfile"><ns1:constraints/><ns1:createdBy>Temporary user handle</ns1:createdBy><ns1:creationTime>YYYY-MM-DDTHH:mm:SS.382+00:00</ns1:creationTime><ns1:description>Allow the datastore to determine the best placement strategy for storage objects</ns1:description><ns1:generationId>0</ns1:generationId><ns1:lastUpdatedBy>Temporary user handle</ns1:lastUpdatedBy><ns1:lastUpdatedTime>YYYY-MM-DDTHH:mm:SS.382+00:00</ns1:lastUpdatedTime><ns1:name>VVol No Requirements Policy</ns1:name><ns1:profileId>fxxxxxxxe-storage-profile-id-5xxxxxxxxxx3</ns1:profileId></ns1:storageProfile>' for '/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk'
YYYY-MM-DDTHH:mm:SS.694Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] DDB: "vvol.FriendlyName" = "FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk" (was "baseDisk-xxxxxx-datastore-attached_3.vmdk")
YYYY-MM-DDTHH:mm:SS.301Z In(166) Hostd[2099590]: [Originator@6876 sub=Libs opID=3xxxxxx4-x1-fx-3xx9 sid=5xxxxxx0 user=vpxuser:Domain\User] OBJLIB-VVOLOBJ : VVolObjGetObjTypeVersion: Returning NS version: 1 for path: /vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk
From the above logs we see;
Current location of the disk:
/vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-name-of-physical-location-in-datastore-xd-delta.vmdk
From the logs the host is looking for the disk at :
OBJLIB-VVOLOBJ : VVolObjGetObjTypeVersion: Returning NS version: 1 for path: /vmfs/volumes/vvol:826xxxxxxxxxxxxx3-9xxxxxxxxxxxxxxxx2/naa.68cxxxxxxxxxxeb5af820xxxxxxxx8c9a/FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk
Difference in the namespace:
version : 1 > FBxxxxxxxxxxxxx01-different-name-from-hostd-logs-updated-disk-xd-delta.vmdk
Updated version : 2 (latest) > FBxxxxxxxxxxxxx01-name-of-physical-location-in-datastore-xd-delta.vmdk
vSphere 8.x
This issue occurs due to VVol storage objects version mismatch between the config vvol namespace of the 2 VM's. The existing VM's config vvol namespace is in the older version, while the upgraded VM's config vvol namespace is at the newer version.
Note: VVols are being deprecated from VCF 9.0, https://knowledge.broadcom.com/external/article?articleNumber=401070
Engineering is aware of this issue on vSphere 8.x, the fix will be release in the next release vSphere 8.0 P07.
Workaround:
- Storage vMotion the VM to another Datastore.
- And Storage vMotion the VM back to the original Datastore
- Now hot add the Existing HDD disk to the VM