Resolving SCSI reservation conflicts

Products

• VMware vSphere ESXi

Issue/Introduction

• You cannot access a LUN.
• The LUN is zoned, presented and configured properly.
• A vdf command never completes.
• You see the error:
  

   

Failure reading partition table from device [naa.xxxxx]: I/O error

Environment

vSphere ESXi 6.x
vSphere ESXi 7.x
vSphere ESXi 8.x

Resolution

 

To identify if SCSI reservation conflicts are preventing a LUN from being accessed:

1. Verify that the LUN is detected by the ESX host at boot time by running the command:
   
   # esxcfg-scsidevs -c
   
   You see output similar to:
   
   Device UID Device Type Console Device Size Plugin Display Name
   eui.################ Direct-Access /dev/sda 70007MB NMP Local FUJITSU Disk (eui.################)
   mpx.vmhba#:C#:T#:L# CD-ROM /dev/sr0 0MB NMP Local HL-DT-ST CD-ROM (mpx.vmhba#:C#:T#:L#)
   mpx.vmhba#:C#:T#:L# Direct-Access /dev/cciss/c0d0 34727MB NMP Local VMware Disk (mpx.vmhba#:C#:T#:L#)
    
2. If the LUN is not listed, rescan the storage. For more information, see Performing a rescan of the storage on an ESX/ESXi host.
3. Check the system logs for signs of SCSI Reservation Conflict errors. Look for lines that indicate if the LUN is having an issue:
   
   cat /var/log/vmkernel.log
   
   Vendor: EMC Model: SYMMETRIX Rev: 5771
   Type: Direct-Access ANSI SCSI revision: 03
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   sdh : READ CAPACITY failed.
   status = c, message = 00, host = 0, driver = 00
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   scsi0 (0,0,52) : RESERVATION CONFLICT
   VMWARE: Device that would have been attached as scsi disk sdh at scsi0,
   channel 0, id 0, lun 52
   Has not been attached because this path could not complete a READ
   command eventhough a TUR worked.
   result = 0x18 key = 0x0, asc = 0x0, ascq = 0x0
   VMWARE: Device that would have been attached as scsi disk sdh at scsi0,
   channel 0, id 0, lun 52
   Has not been attached because it is a duplicate path or on a passive
   path
   scan_scsis starting finish
   scan_scsis done with finish
   
   Note: In this example, LUN 52 is inaccessible in the host cluster. Since it is listed in the output of step 1, it was accessible at some point (a host reserved the LUN and never released it, possibly due to a SAN switch reboot in the middle of the reservation operation).

To resolve this:

1. Examine all hosts (also applies to hosts for any pending reservations) by running the command:
   
   # esxcfg-info | egrep -B5 "s Reserved|Pending"
   
   You see output similar to:
   
   Note: Not all of the output is shown.
   
   ----Console Device....................../dev/sdd
   |----DevfsPath.......................... /vmfs/devices/disks/vml.######################################################
   |----SCSI Level..........................6
   |----Queue Depth.........................128
   |----Is Pseudo...........................false
   |----Is Reserved.........................false
   |----Pending Reservations................0
   --
   |----Console Device....................../dev/sda
   |----DevfsPath........................../vmfs/devices/disks/vml.######################################################
   |----SCSI Level..........................6
   |----Queue Depth.........................128
   |----Is Pseudo...........................false
   |----Is Reserved.........................false
   |----Pending Reservations................ 1
   
   Note: The host that has Pending Reserves with a value that is larger than 0 is holding the lock.
    
2. Perform a LUN reset to clear the lock by running the command:
   
   # vmkfstools --lock lunreset /vmfs/devices/disks/vml.######################################################
    
3. Verify that the LUN no longer has any Pending Reserves by running the command:
   
   # esxcfg-info -s | egrep -B16 "s Reserved|Pending"
   
   Note: If the Pending Reserves is not 0 or Is Reserved is not false, the LUN reset did not succeed.
    
4. Do a refresh of the storage window on each ESXi/ESX host.
5. If the datastore is not mounted, perform a rescan. For more information, see Performing a rescan of the storage on an ESX/ESXi host .

Note: Virtual machines housed on a LUN should observe no interruption in service when a LUN reset occurs.

Additional Information

 

Troubleshooting LUN connectivity issues on ESXi hosts