After vMotion, WFSC RDM Disks Go Offline with "Matching reservation key not found" Error
search cancel

After vMotion, WFSC RDM Disks Go Offline with "Matching reservation key not found" Error

book

Article ID: 448166

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

 

  • Virtual machines participating in a Windows Server Failover Cluster (WSFC) experience a storage disruption immediately following a live migration (vMotion).

  • The vMotion migration completes successfully from a compute perspective, but the destination ESXi host fails to acquire the necessary SCSI locks on the underlying Raw Device Mapping (RDM) disks.

  • In the clustering configuration, drives report an offline state and access is lost.

  • A full virtual machine reboot is required to clear the failure state and bring the disks back online.

  • In /var/run/log/vmkernel.log. below events will be seen on the destination host

    2026-05-18T02:02:26.230Z In(182) vmkernel: cpu43:3337398)VSCSI: 2504: handle 14334028152841160(GID:9160)(vscsi1:0):VSCSI_RestoreResv resKey 4d730103d2a9b411, isReserved 1 resvType 3 type 5 isSource 0 scope 0 aptpl 0 spec_i_pt 0 all_tg_pt 0
    2026-05-18T02:02:26.231Z In(182) vmkernel: cpu43:3337398)VSCSI: 2537: handle 14334028152841160(GID:9160)(vscsi1:0):Matching reservation key not found during VMotion.
    2026-05-18T02:02:26.231Z In(182) vmkernel: cpu43:3337398)VSCSI: 2504: handle 14334028157035465(GID:9161)(vscsi1:1):VSCSI_RestoreResv resKey 4d730103d2a9b411, isReserved 1 resvType 3 type 5 isSource 0 scope 0 aptpl 0 spec_i_pt 0 all_tg_pt 0
    2026-05-18T02:02:26.231Z In(182) vmkernel: cpu43:3337398)VSCSI: 2537: handle 14334028157035465(GID:9161)(vscsi1:1):Matching reservation key not found during VMotion.
    2026-05-18T02:02:26.231Z In(182) vmkernel: cpu43:3337398)VSCSI: 2504: handle 14334028161229770(GID:9162)(vscsi1:2):VSCSI_RestoreResv resKey 4d730103d2a9b411, isReserved 1 resvType 3 type 5 isSource 0 scope 0 aptpl 0 spec_i_pt 0 all_tg_pt 0
    2026-05-18T02:02:26.231Z In(182) vmkernel: cpu43:3337398)VSCSI: 2537: handle 14334028161229770(GID:9162)(vscsi1:2):Matching reservation key not found during VMotion
  • Also, destination host fails to acquire the reservation on the device

    cat /storage/scsifw/devices/naa.######################/info 
    Device information {
       owning plugin :NMP
       state : 0 -> on
       APD reason : 1 -> none
       device type : 0 -> Direct-Access
       partition table type : 2 -> gpt
       vendor :3PARdata
       model :VV
       revision :4620
       capacity:info {
          logical block size :512
          num of logical blocks :2147483648
          physical block size :512
          first aligned LBA over PBA :0
       }
       Device Format: 1 -> 512n
       Max Transfer Number of blocks:32768
       SCSI version:0x06
       DevFS registration type: 1 -> Disk devfs registration
       is pseudo-device?:0
       is device reserved?:0  ---> this should ideally be 1 after the vm gets migrated
       is device local?:0
       is device SSD?:1
       is device NVME?:0
       is LUN queue depth being throttled?:0
       is Volatile Writable Cache Supported?:1
       is Volatile Writable Cache Enabled?:0
       is Protocol Endpoint?:0
       is Emulated DIF/DIX Enabled?:0
       format progress %:4294967295
       format result: 0 -> unknown
       Thin provisioning status: 1 -> yes
       device availability: 3 -> redundant availability
       # pending reservations:0
       current open count:1
       current schedQ count:3
       ActiveQCount:0
       VAAI clone status: 1 -> supported
       VAAI zero status: 1 -> supported
       VAAI delete status: 1 -> supported
       VAAI ATS status: 1 -> supported
       VAAI Extended clone status: 2 -> unsupported
       PI:Protection info {
          Is protection enabled on this device:0
          Is PI supported by the device:0
          Is PI activated for the device:0
          Protection Type:0
          Protection Mask:0x00000000
          Supported Guard: 0 -> NO GUARD SUPPORT
          Is DIX Enabled:0
          DIX Guard: 0 -> NO GUARD SUPPORT

 

Environment

VMware Esxi (All versions)

Cause

  • Storage spaces is a virtual layer on top of disks Esxi present to WFSC so ESX will not be able to manage it. When storage spaces feature is used, the guest sends REGISTER commands which are not supported by ESX. The ESXi virtual SCSI layer (vSCSI) code is designed to track cluster keys via REGISTER_AND_IGNORE (0x6) commands. It does not intercept or tack REGISTER(0x0) commands. 

  • When vMotion initiates, the source host packages up its stale tracked key and ships it to the destination host. The destination host queries the disk, compares the real key against the stale key it was sent , realizes they do not match, and fails to reserve the disk.

Resolution

It is recommended to present the storage disks directly to the Windows guest operating system using either physical pass-through RDMs or clustered shared VMDKs. Do not create Microsoft Storage Spaces on top of the devices, as this is not a supported configuration for ESXi with WSFC.