VM Backups with HotAdd transport mode fails on VVOL datastores
search cancel

VM Backups with HotAdd transport mode fails on VVOL datastores

book

Article ID: 440635

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

  • VM backups run with HotAdd transport mode fails with following error:

    “Failed to add disk scsi0:#. One of the parameters supplied is invalid…”
    Error: Failed to add disk scsi0:#. Failed to lock the file Cannot open the disk '/vmfs/volumes/vvol:#################-################rfc####.#######-####-####-####-############/<VM-name>.vmdk' or one of the snapshot disks it depends on. Failed to power on scsi0:#.

  • Manual adding of VM disk to be backed up fails with error:

    Failed to add disk scsi0:#. Failed to lock the file Cannot open the disk '/vmfs/volumes/vvol:#################-################rfc####.#######-####-####-####-############/<VM-name>.vmdk' or one of the snapshot disks it depends on. Failed to power on scsi0:#.


  • vmware.log reports following error after adding the virtual disk and on trying to open it:

    2026-06-09T19:45:23.578Z In(05) vmx 2331b6e3-01-45-924b HotAdd: Adding scsi-hardDisk with mode 'independent-nonpersistent' to scsi0:2
    2026-06-09T19:45:23.578Z In(05) vmx 2331b6e3-01-45-924b DISK: OPEN scsi0:2 '/vmfs/volumes/vvol:###################-#####################/rfc4122.########-####-####-####-############/<VM-000001.vmdk>' independent-nonpersistent R[]
    2026-06-09T19:45:23.581Z In(05) vmx 2331b6e3-01-45-924b AIOGNRC: Failed to open '/vmfs/devices/vvol/rfc4122.########-####-####-####-############' : Could not find the file (60003) (0x4011).
  • Issue first observed on fresh deployment to 8.0U3 and is also seen post upgrade to 9.0 version.

  • Pure Arrays is the storage configured for vvols. No issues is observed in another environment running on 7.0 version.

  • On initiating backup with HotAdd option, snapshot creation completes successfully. However, Reconfigure task fails (it attempts four times and then backup terminates.)

  • Backups conducted with NBD option are successful but is tremendously slow as it uses NFC protocol.

Environment

  • VMware vSphere 8.x

  • VMware vSphere 9.x

  • VMware Cloud Foundation 9.x

 

Cause

  • A disk backed up on ESXi 7.0U3 uses Namespace (NS) version 1. When restoring this VM on an ESXi 8.x host, the environment defaults to NS version 2.  The operation fails during the hot-add of the restored disk on 8.0+ (specifically observed with Non-Persistent Independent (NPI) disks hot add) because the descriptor created are having NS 2 while the VM already existed is from NS 1 (Created in 7.0U3 or before timeframe) where hot add being done.


  • Also, (in vmware.log of VM to be backed up) update of descriptor fails as shown below:

    YYYY-MM-DDTHH:MM:SS.497Z In(05) vmx 2331b6e3-01-45-924b OBJLIB-VVOLOBJ :VVolObjOpen: Object opened with flags - 20000
    YYYY-MM-DDTHH:MM:SS.497Z In(05) vmx 2331b6e3-01-45-924b OBJLIB-VVOLOBJ : VVolObjGetSizeFromDDB: Could not find key Size_CommittedLastUpdateTime, populating cache again.
    YYYY-MM-DDTHH:MM:SS.535Z In(05) vmx 2331b6e3-01-45-924b 2157558:VVOLLIB : VVolLibSpaceStatsInt:6429: Successful, returns object(s) 1
    YYYY-MM-DDTHH:MM:SS.539Z In(05) vmx 2331b6e3-01-45-924b DISKLIB-VMFS  : "vvol://################-################/rfc####.########-####-####-####-############" : open successful (1) size = 42949672960, hd = 0.
     Type 3
    YYYY-MM-DDTHH:MM:SS.539Z In(05) vmx 2331b6e3-01-45-924b OBJLIB-VVOLOBJ : VVolObjGetObjTypeVersion: Returning NS version: 1 for path: ./<VM.vmdk>.REDO_012o7v
    YYYY-MM-DDTHH:MM:SS.539Z Er(02) vmx 2331b6e3-01-45-924b DISKLIB-DSCPTR: DescriptorWriteInt: Failed to update the descriptor version of './<VM.vmdk>.REDO_012o7v': One of the parameters supplied is invalid

Resolution

This is a known issue and Broadcom is aware of it:

    1. For vSphere 8.x -- Fix is available in 8.0.3 P10

    2. For vSphere 9.x (VCF 9.x) -- Fix is available in 9.1 GA and later versions.