Consolidating disks associated with a backup snapshot fails with the error: The file is already in use

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• When consolidating the disks associated with a backup snapshot, you receive the error:
  
  The file is already in use
  
  
• After enabling increased logging, in the hostd.log file you see messages similar to:
  
  
  • [2C581B90 info 'BlklistsvcPlugin' opID=9e67f392-59] Block list built for [] /vmfs/volumes/50194e8a-f6a02118-9f78-90e2ba0fa95c/example_vm/example_vm.vmdk; lease is 528d29b7-c357-8fd8-1c38-9ed9746a0dcc
    
    
  • [4B6B3B90 verbose 'vm:/vmfs/volumes/50648696-74a99014-b76f-14feb58c727a/example_vm/example_vm.vmx'] Done disk consolidation.
[4B6B3B90 info 'vm:/vmfs/volumes/50648696-74a99014-b76f-14feb58c727a/example_vm/example_vm.vmx'] Failed to consolidate disks in Foundry: Error: (15) The file is already in use
    
    
  • [4B6B3B90 verbose 'vm:/vmfs/volumes/50648696-74a99014-b76f-14feb58c727a/example_vm/example_vm.vmx'] Done disk consolidation.
[4B6B3B90 info 'vm:/vmfs/volumes/50648696-74a99014-b76f-14feb58c727a/example_vm/example_vm.vmx'] Failed to consolidate disks in Foundry: Error: (15) The file is already in use

Environment

Virtual Disk Development Kit 5.1
Virtual Disk Development Kit 5.0
VMware Virtual Disk Development 1.2

Cause

This issue occurs because the consolidation attempt is made prior to the backup proxy lease being cleaned up.

The series of events should match:

1. Backup starts
2. Snapshot created for backup
3. Lease established
4. Failure of SAN mode, then fallback to NBD mode
5. Backup finishes
6. Consolidation attempt (fails)
7. Lease released

Note: The backup proxy must be a separate physical machine to trigger the issue; virtual machine based proxies cannot access SAN mode backups.

Resolution

This is a known issue affecting Virtual Disk Development Kit (VDDK) 5.0 and 5.1. The issue is fixed in VDDK 5.1.2, 5.5 and in 5.0 Update 3.

After the lease is released, normal consolidation is possible. For more information on cleaning up the virtual machine state, see Consolidating snapshots in vSphere 5.x (2003638).

To work around this issue, configure the backup software to attempt NBD directly instead of attempting SAN-based backup first, or ensure the correct block devices are present on the backup proxy server prior to attempting a SAN-based backup.

Note: Auto mode attempts SAN-based backup first then falls back to NBD if that fails. This situation will also trigger the issue.

Additional Information

How to consolidate snapshots in vSphere 5.x/6.x
Snapshot consolidation after backing up a virtual machine fails with the error: Failed to consolidate disks in Foundry: Error: (15) The file is already in use
バックアップ スナップショットと関連付けられたディスクの統合が次のエラーで失敗する: ファイルはすでに使用されています
整合与备份快照关联的磁盘失败并显示以下错误：文件已在使用中