ISCSI storage not seeing all adapter cards for iSCSI.

Products

• VMware vSphere ESXi

Issue/Introduction

A vSphere Administrator finds that they are unable to ping the iSCSI array from the command line of a 3 node ESXi host cluster.

Environment

 Esxi 7.0p03, UCS C240 M5S SN: WZP2515009Y, Cisco Vic Fnic Controller, Pure iSCSI array.

Cause

Gs Worked to identify a misconfiguration of the internal broadcast network which impaired the hosts ability to communicate with the iSCSI Storage array.  

Through a series of ping tests utilizing vmkping -I vmk3 ##.#.###.#0 GS was able confirm the three standalone hosts were unable to ping one another reliably.  Multiple ping tests to customer provided IP ranges demonstrated the hosts had limited communication scope to one another and the array (broadcast/Subnet config).

Once confirming the hosts were not able to reliably ping one another, the next step is to review the host network configuration using ESXi tools esxfg-vmnics -l, esxcfg-vswitch -l, and esxcfg-vmknic -l.

In isolating the two ports on the adapter in question, GS was able to confirm that while both adapters could ping their internal IPs ##.#.###.#4 and ##.#.###.#5, but only first port (from host1) could ping the iSCSI array as well as a limited number of ports on the other hosts.  The second port on the card was unable to communicate externally to any hosts/upstream components.

This led GS to the conclusion that the customer internal broadcast domain was misconfiguration or there was a hardware issue upstream.

For further troubleshooting help see the following troubleshooting article - Testing VMkernel network connectivity with the vmkping command

 

 

Resolution

It is recommended the local networking Team review their internal adapter configuration to unsure their subnet is configured to broadcast properly, as external components are not reachable.  Additionally the administrator should review their array SP configuration to ensure they can ping to the host nics appropriately.