RDM Disk Attachment Failure Due to VMFS Locking Conflict in virtual machine

Products

VMware vCenter Server VMware vSphere ESXi 8.0 VMware vCenter Server 8.0

Issue/Introduction

Following failed task will be generated when application cluster is trying to attach/map RDM disks to the VM -

Task Name
Reconfigure virtual machine

Status     
    Unable to access file  since it is locked
    filePath: /vmfs/volumes/<datastore-uuid>/<vm-name>/<vm-name>-rdmp.vmdk
    lockMode: 0
    mac: 00000000-00000000-0000-000000000000
    Locking conflict for file "/vmfs/volumes/<datastore-uuid>/<vm-name>/<vm-name>-rdmp.vmdk". VMFS on-disk lock owner is host with MAC address 00000000-00000000-0000-000000000000 and VMFS lock mode 0x0.
    File system specific implementation of Ioctl[file] failed
    File system specific implementation of OpenFile[file] failed
    File system specific implementation of OpenFile[file] failed
    File system specific implementation of OpenFile[file] failed
    File system specific implementation of OpenFile[file] failed
    File system specific implementation of OpenFile[file] failed
    File system specific implementation of OpenFile[file] failed
    Failed to lock the file
    Cannot open the disk '/vmfs/volumes/<datastore-uuid>/<vm-name>/<vm-name>.vmdk' or one of the snapshot disks it depends on.
    Failed to power on scsi1:4.
    Failed to add disk scsi1:4.
   
Initiator
administrator@vsphere.local
Target
########## Navigates to Virtual machine
Server
<VCSA FQDN> Navigates to vCenter server
Details
Reconfiguring Virtual Machine on destination host
Error stack:
filePath: /vmfs/volumes/<datastore-uuid>/<vm-name>/<vm-name>-rdmp.vmdk
lockMode: 0
mac: 00000000-00000000-0000-000000000000
Locking conflict for file "/vmfs/volumes/<datastore-uuid>/<vm-name>/<vm-name>-rdmp.vmdk". VMFS on-disk lock owner is host with MAC address 00000000-00000000-0000-000000000000 and VMFS lock mode 0x0. 
File system specific implementation of Ioctl[file] failed
File system specific implementation of OpenFile[file] failed
File system specific implementation of OpenFile[file] failed
File system specific implementation of OpenFile[file] failed
File system specific implementation of OpenFile[file] failed
File system specific implementation of OpenFile[file] failed
File system specific implementation of OpenFile[file] failed
Failed to lock the file
Cannot open the disk '/vmfs/volumes/<datastore-uuid>/<vm-name>/<vm-name>.vmdk' or one of the snapshot disks it depends on. 
Failed to power on scsi1:4. 
Failed to add disk scsi1:4.

Failed task from vSphere Client-

Environment

VMware vSphere ESXi 8.0 (VMware vSphere ESXi 8.0)
VMware vCenter Server 8.0 (VMware vCenter Server 8.0)

Cause

This issue is caused due to errors at the associated physical LUN from where the RDM disk is crabbed out, that fails to get attach or map to the VM.
Reference from storage array controller (ontap or system manager from Netapp)-
A remote cluster or remote storage VM could not be reached or a relationship was deleted but not released.

Steps to validate the error from ontap/netapp storage appliance-

- Login to Ontap System Manager
- Go to Storage and then LUNs
- Select the LUN/datastore from where the RDM disks are crabbed out
- Check Overview for the errors/status/warnings

Resolution

Engage storage team/venvdor to ensure cluster stability and prevent locking conflicts, the configuration of shared RDMs and shared disks must be identical across all nodes from backend storage LUN/array

Additional Information

Refer some of the associated articles related to this issue -
A remote cluster or remote storage VM couldn't be reached or a SnapMirror replication relationship was deleted but not released

Resolving file locking conflicts and unexpected reboots while adding shared disk to the secondary node

Raw Device Mapping in vSphere