Rubrik backup failures - no access to volumes

Products

• VMware vSphere ESXi

Issue/Introduction

Rubrik was having backup failures with the following error "Vmware.UnableFetchDiskData"

 

Cause

The storage array vendor is creating snapshot LUNs as part of the storage level process, then removing them when it is done using them. Rubrik is then attempting to back-up LUNs that no longer exist as they were removed from the hosts by the array vendor

Datastores are seen as snapshots

From the vmkernel log

LVM: 12010: Device naa.######################### detected to be a snapshot:
LVM: 12016: queried disk ID: <type 2, len 22, lun 4, devType 0, scsi 0, h(id) 8435431782611819699>
LVM: 12023: on-disk disk ID: <type 2, len 22, lun 243, devType 0, scsi 0, h(id) 16734552433555716387>
LVM: 12040: If Device naa.######################### is OsData:0 status:Success

In the vobd log we then see the devices being unpresented to the hosts

[vob.scsi.scsipath.pathstate.dead] scsiPath vmhba2:C0:T4:L1 changed state from on
[esx.problem.storage.connectivity.lost] Lost connectivity to storage device naa.#########################. Path vmhba2:C0:T4:L1 is down. Affected datastores: Unknown.
[vob.scsi.scsipath.remove] Remove path: vmhba2:C0:T4:L1

 

Resolution

Contact storage array vendor to reconfigure so array level snapshots are not presented to the ESXi hosts.

Additional Information

Troubleshooting Storage Array Integrated VM Backups For Rubrik / Veeam