After a reboot of an ESXi host, VMFS datastores are no longer visible

Products

• VMware vSphere ESXi
• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x

Issue/Introduction

Lost access to storage device originating from a single array.
Some hosts you only have access though one side of the fabric 
The issue may be seen on different clusters, different interconnects, different blades and different rack mounts.

 

Environment

VMware vSphere ESXi 7.0
VMware vSphere ESXi 8.0

Cause

On reboot PLOGi may fail for a specific storage target.

• In this case /var/log/vmkernel.log will report sucessful fabric login (FLOGI). eg

 ESXi : /var/log/vmkernel.log
####-##-##T##:##:##.###Z cpu#:2098354)nfnic: <1>: INFO: fdls_send_fabric_flogi: 967: Sending fabric FLOGI for wwpn:0x###############1 Setting FLOGI MFS to 2112
####-##-##T##:##:##.###Z cpu#:2098361)nfnic: <2>: INFO: fdls_send_fabric_flogi: 967: Sending fabric FLOGI for wwpn:0x###############2 Setting FLOGI MFS to 2112
####-##-##T##:##:##.###Z cpu#:2098355)nfnic: <1>: INFO: fdls_process_flogi_rsp: 440: FLOGI response accepted: 0x######
####-##-##T##:##:##.###Z cpu#:2098355)nfnic: <-1>: INFO: fnic_queue_wq_copy_desc_flogi_reg: 197: sending FLOGI registration
####-##-##T##:##:##.###Z cpu#:2098392)nfnic: <1>: INFO: fnic_fcpio_flogi_reg_cmpl_handler: 1197: FLOGI reg succeeded
####-##-##T##:##:##.###Z cpu#:2098355)nfnic: <1>: INFO: fnic_fdls_register_portid: 2001: FLOGI registration success
####-##-##T##:##:##.###Z cpu#:2098362)nfnic: <2>: INFO: fdls_process_flogi_rsp: 440: FLOGI response accepted: 0x######
####-##-##T##:##:##.###Z cpu#:2098362)nfnic: <-1>: INFO: fnic_queue_wq_copy_desc_flogi_reg: 197: sending FLOGI registration
####-##-##T##:##:##.###Z cpu#:2098092)nfnic: <2>: INFO: fnic_fcpio_flogi_reg_cmpl_handler: 1197: FLOGI reg succeeded
####-##-##T##:##:##.###Z cpu#:2098362)nfnic: <2>: INFO: fnic_fdls_register_portid: 2001: FLOGI registration success

 ESXi : /var/log/vmkernel.log then reports PLOGI (Port Login) fails due to busy target status.
####-##-##T##:##:##.###Z cpu#:2098355)nfnic: <1>: INFO: fdls_process_tgt_plogi_rsp: 1706: PLOGI returned FC_LS_REJ from target: 0x####1a
####-##-##T##:##:##.###Z cpu#:2098355)nfnic: <1>: INFO: fdls_process_tgt_plogi_rsp: 1706: PLOGI returned FC_LS_REJ from target: 0x####2a

####-##-##T##:##:##.###Z cpu##:2098362)nfnic: <2>: INFO: fdls_process_tgt_plogi_rsp: 1699: PLOGI returned FC_LS_REJ BUSY from target. Will retry from timer routine: 0x####1a
####-##-##T##:##:##.###Z cpu##:2098362)nfnic: <2>: INFO: fdls_process_tgt_plogi_rsp: 1699: PLOGI returned FC_LS_REJ BUSY from target. Will retry from timer routine: 0x####2a

 

 

Resolution

• Please contact your storage vendor to investigate further, as the PLOGI events are being rejected by the target.
• In large environments, please ensure that you have not reached the any storage array limitations in regards to how may initiators can be logged into the SAN.