RecoverPoint for VMs replication fails on vSphere with failure to open vRPA journal volume

Products

• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x

Issue/Introduction

• RecoverPoint for VMs replication of VMs in a vSphere environment with failure to lock/open vRPA (Virtual RecoverPoint Appliance) journal vmdk disk.
• /var/log/vmkernel.log has logging similar to:
  

  YYYY-MM-DDTHH:MM:SS.956Z cpu44:2105137)NFSLock: 5032: failed to get lock on file ##############_IOFilter_JVOL_00288-flat.vmdk(.lck-7412150000000000) 0x430b567efdf0 on ##########  ###.###.#.# (###.###.##.##): Busy
  YYYY-MM-DDTHH:MM:SS.874Z cpu74:3933292)SchedVsi: 2083: Group: host/vim/vmvisor/ntnx(2268): min=0 max=200 minLimit=unlimited, units: mb

• In a vSAN environment, in addition, the following may be logged in /var/log/iofilterd-emcjiraf.log  
  

  YYYY-MM-DDTHH:MM:SS.517Z iofilterd-emcjiraf[2104572]: FiltLib: Opening virtual disk ("/vmfs/volumes/vsan:######################################/########/#############_IOFilter_JVOL_00668.vmdk") failed: "Failed to lock the file" (16392).
  YYYY-MM-DDTHH:MM:SS.517Z iofilterd-emcjiraf[2104572]: RPVS_iof_LunInfo_OpenLun: Failed to open vmdk file of lun 668 with status Busy path /vmfs/volumes/vsan:#####-######-#####-#######/#######/###################_IOFilter_JVOL_00668.vmdk
  YYYY-MM-DDTHH:MM:SS.517Z iofilterd-emcjiraf[2104572]: execute_open_cmd: Cluster id: 0x5d0aa2fff4d45cf4 lun 668 open failed. open_rc: Busy con 0xc6db9c9340
  YYYY-MM-DDTHH:MM:SS18Z iofilterd-emcjiraf[2104572]: execute_close_cmd: cluster id: 0x5d0aa2fff4d45cf4 lun 668 is not opened (openState: 0). Ignore close request
  YYYY-MM-DDTHH:MM:SS.519Z iofilterd-emcjiraf[2104572]: execute_close_cmd: cluster id: 0x5d0aa2fff4d45cf4 lun 48 is not opened (openState: 0). Ignore close request
  YYYY-MM-DDTHH:MM:SS.519Z iofilterd-emcjiraf[2104572]: execute_close_cmd: cluster id: 0x5d0aa2fff4d45cf4 lun 507 is not opened (openState: 0). Ignore close request
  YYYY-MM-DDTHH:MM:SS.519Z iofilterd-emcjiraf[2104572]: execute_close_cmd: cluster id: 0x5d0aa2fff4d45cf4 lun 558 is not opened (openState: 0). Ignore close request
  YYYY-MM-DDTHH:MM:SS.520Z iofilterd-emcjiraf[2104572]: execute_close_cmd: cluster id: 0x5d0aa2fff4d45cf4 lun 359 is not opened (openState: 0). Ignore close request
  YYYY-MM-DDTHH:MM:SS.228Z iofilterd-emcjiraf[2104572]: OBJLIB-VSANOBJ: VsanObjLock: Object '#######-#######-######-########' locked, retrying.
  YYYY-MM-DDTHH:MM:SS.581Z iofilterd-emcjiraf[2104572]: OBJLIB-VSANOBJ: VsanObjLock: Object '#######-#######-######-########' locked, retrying.

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

RecoverPoint for Virtual Machines uses journal volumes to track and manage point in time images. If a vRPA appliance is vMotioned to a different ESXi host, Data Replication Unavailable(DRU), can occur. For further details, see RecoverPoint for VMs: Journal Volumes Locked to Specific ESXi Hosts Can Lead to Data Replication Unavailable and Reboot Regulation

Resolution

  Please contact Dell for further assistance. 

Additional Information

Note: Ensuring that the the vRPA VM is not part of a DRS configuration and is not migrated to another host may help avoid this issue.