Cannot connect to storage across multiple vSphere Clusters
search cancel

Cannot connect to storage across multiple vSphere Clusters

book

Article ID: 431386

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms: 

  • vCenter event logs shown

Path redundancy to storage device naa.######################84a1d degraded. Path vmhba1:C#:T#:L# is down. Affected datastores: VMFS-Datastore.

  • RSCN (State Change Notification) indicates the change in fabric in # localcli san fc event list

2026-02-20 04:39:00.306 [vmhba1] RSCN received for PID 31b0a8
2026-02-20 04:39:00.306 [vmhba1] RSCN received for PID 31b0a7

  • Host experienced the device loss in /var/run/log/vmkernel.log

2026-02-20T04:39:00.309Z Wa(180) vmkwarning: cpu59:2098121)WARNING: lpfc: lpfc_start_devloss:4565: vmhba1 3248 Start 10 sec devloss tmo WWPN 20:f#:d#:3#:ea:5#:8#:a# NPort x31b0ab
2026-02-20T04:39:00.329Z Wa(180) vmkwarning: cpu59:2098121)WARNING: lpfc: lpfc_start_devloss:4565: vmhba1 3248 Start 10 sec devloss tmo WWPN 20:f#:d#:3#:ea:5#:8#:a# NPort x31b0aa
2026-02-20T04:39:00.344Z Wa(180) vmkwarning: cpu59:2098121)WARNING: lpfc: lpfc_start_devloss:4565: vmhba1 3248 Start 10 sec devloss tmo WWPN 20:f#:d#:3#:ea:5#:8#:a# NPort x31b165

  • Host experienced the transient errors 

2026-02-20T04:39:00.539Z In(182) vmkernel: cpu53:2098433)NMP: nmp_ThrottleLogForDevice:3898: H:0xc D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x430f25a12e80 CmdSN 0x3a0
2026-02-20T04:39:00.570Z In(182) vmkernel: cpu53:2098433)ScsiDeviceIO: 4633: Cmd(0x45da251b9700) 0x8a, CmdSN 0x3a0 from world 4494813 to dev "naa.######################84a1d" failed H:0xc D:0x0 P:0x0
2026-02-20T04:39:00.574Z In(182) vmkernel: cpu53:2098433)ScsiDeviceIO: 4633: Cmd(0x45da250ff100) 0x8a, CmdSN 0x3a0 from world 4494813 to dev "naa.######################84a1d" failed H:0xc D:0x0 P:0x0

  • PRLI failed 

2026-02-20T04:48:42.118Z In(182) vmkernel: cpu60:2098121)lpfc: lpfc_cmpl_els_prli:2431: vmhba1 0103 PRLI completes to NPort x310043 Data: x3 x103 x14 x0
2026-02-20T04:48:42.118Z In(182) vmkernel: cpu60:2098121)lpfc: lpfc_els_retry:4864: vmhba1 0108 No retry ELS command x14001020 to remote NPORT x310043 Retried:1 Error:x3/x103
2026-02-20T04:48:42.118Z Wa(180) vmkwarning: cpu60:2098121)WARNING: lpfc: lpfc_cmpl_els_prli:2451: vmhba1 2754 PRLI failure DID:310043 Status:x3/x103
2026-02-20T04:48:42.118Z In(182) vmkernel: cpu60:2098121)lpfc: lpfc_cmpl_els_prli:2431: vmhba1 0103 PRLI completes to NPort x310023 Data: x3 x103 x14 x0

Environment

vSphere ESXi 8.0 
vSphere ESX 9.0 

Cause

The switch issued frequent RSCNs during the incident, VMHBA experienced frequent link fluctuations . This instability triggered multiple 'Path Redundancy Lost', ultimately resulting in datastore connectivity issues on the host.

  • Fluctuating Link for vmhba1 alone 

# localcli san fc event list

2026-02-20 04:39:27.791 [vmhba1] LINK DOWN
2026-02-20 04:39:29.062 [vmhba1] LINK UP
[cont..]
2026-02-20 04:50:37.800 [vmhba1] LINK DOWN
2026-02-20 04:59:50.930 [vmhba1] LINK UP

Resolution

Since the cause of this issue is external to the ESXi hosts. This must be investigated at the SAN switch level to identify any fabric-side disruptions.