Lost connectivity to datastore due to iSCSI Nop-out timeouts

Products

• VMware vCenter Server

Issue/Introduction

/var/run/log/syslog.log

####-##-##T##:##:##Z iscsid [2098919}: connection 1:0 (iqn.####-##.###.#####################) if =default addr=###.##.##.##:3260 (TPGT:#### ISID:0x1) (T# C#)) Nop-out timeout after 10 sec in state (3)

####-##-##T##:##:##Z iscsid: connection #:# (eui.################# if=iscsi_vmk@vmk1 addr=###.##.##.##:3260 (TPGT:#### ISID:0x1)  (T# C#)) Nop-out timeout after 10 sec in state (3).

Lost connectivity to device <Device ID> backing the boot filesystem. /vmfs/devices/disks/<Device ID>As a result, the host configuration changes will not be saved to persistent storage.

Intermittent loss of connectivity to a VMFS iSCSI based Datastore. 

 

Environment

• VMware vSphere ESXi 6.x
• VMware vSphere ESXi 7.x
• VMware vSphere ESXi 8.x

Cause

A NOP-Out timeout occurs if NOP-Outs do not receive a response from the ISCSI target for a period of 10 seconds

 

NOP-Out is used by the initiator as kind of "ping request" at the iSCSI layer to verify that a connection/sessions are still active and all its components are operational and active. 

This operation (NOP OUT) is performed by the initiator to the target.  The NOP-In response is the "ping echo".

Resolution

In order to investigate and resolve the Nop-out timeouts please engage with your Network and storage vendor.