Expanding a VVOL Hard Disk Fails with Error : "A specified parameter was not correct: spec.deviceChange.device. The disk extend operation failed: One of the parameters supplied is invalid"

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Expansion of hard disk through both vCenter and Host Client on a VVOL datastore fails with error : "A specified parameter was not correct: spec.deviceChange.device. The disk extend operation failed: One of the parameters supplied is invalid".

Validation Step:

•  Expansion of hard disk via CLI fails with error :
  
  "OBJLIB-VVOLOBJ : VVolObjTruncateInt: Failing resize request for vvol, as it has referencesContainerId=xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx, VVolId=yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy Failed to extend disk : One of the parameters supplied is invalid (1)".

 

Environment

VMware vSphere ESXi 8.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 6.x

Cause

When a snapshot is taken on vvol, both the array and vSphere maintain records of it. Upon snapshot deletion, vSphere sends a delete request to the VVOL array but the array does not confirm it. This can result in a stale snapshot, which appears as an "Empty response".

Cause Validation: 

• In the /vmfs/volume/datastoreUUID/VM/vmware.log file, similar entries are seen:
  
  YYYY-MM-DDTHH:MM.SSSZ Wa(03) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 OBJLIB-VVOLOBJ : VVolObjTruncateInt: Failing resize request for vvol, as it has referencesContainerId=xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx, VVolId=yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy
YYYY-MM-DDTHH:MM.SSSZ In(05) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 OBJLIB-VVOLOBJ : VVolObjTruncate: Failed to resize VVolId='yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' to size 966367641600 (Oneof the parameters supplied is invalid)
YYYY-MM-DDTHH:MM.SSSZ In(05) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 OBJLIB-LIB: ObjLib_Truncate: Failed to truncate handle 26: One of the parameters supplied is invalid (0x9).
YYYY-MM-DDTHH:MM.SSSZ Er(02) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 DISKLIB-FLAT  : ObjLib_TruncateByObjID failed: One of the parameters supplied is invalid (9)
YYYY-MM-DDTHH:MM.SSSZ Er(02) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 DISKLIB-LINK  : DiskLinkGrow: Failed to create extent 'vvol://xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx/yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy: One of the parameters supplied is invalid
YYYY-MM-DDTHH:MM.SSSZ Er(02) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 DISKLIB-LIB   : Unable to grow disk '/vmfs/volumes/vvol:xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx/yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/VM_1.vmdk' : One of the parameters supplied is invalid (1).
YYYY-MM-DDTHH:MM.SSSZ Er(02) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 DISKLIB-LIB   : DiskLibGrowInt: Unable to grow disk '/vmfs/volumes/vvol:xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx/yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/VM_1.vmdk' : One of the parameters supplied is invalid (1).
YYYY-MM-DDTHH:MM.SSSZ Er(02) worker-140161507 m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 DISKLIB-LIB   : Failed to grow disk '/vmfs/volumes/vvol:xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx/yyyyyyy.yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/VM_1.vmdk' : One of the parameters supplied is invalid (1).
YYYY-MM-DDTHH:MM.SSSZ Er(02) vmx m35a45sk-1222150-auto-q70n-h5:70197682-eb-01-15-33c9 HotExtendPreGrowWorkerCompletionCB: DiskLib_PreGrowExtent failed: One of the parameters supplied is invalid (1)
  
  
• In the /var/run/log/vvold.log file, similar entries are seen: 
  
  YYYY-MM-DDTHH:MM.SSSZ Er(163) VVold[2099472]: [Originator@6876 sub=IpcSvc IpcCtx=2101469-14] HandleRequest: Error handling function QueryUuidForFriendlyName: Empty response
YYYY-MM-DDTHH:MM.SSSZ In(166) VVold[2099510]: [Originator@6876 sub=Default] SI:QueryIncompleteVirtualVolumes@5743 Empty response For Container: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Resolution

Engage storage vendor and clean up these stale snapshots left on the array.

Workaround:

Clone the virtual machine.