Datastores disappear after patching ESXi hosts
Article ID: 375516
Updated On:
Products
Issue/Introduction
Datastores presented to ESXi hosts from the storage arrays may go missing after you patch, update or upgrade your ESXi hosts. This is a common problem and this article shows what you need to do in such a scenario.
1. You don't see any other datastore besides the local datastore. All datastores presented from the storage array are missing.
2. You don't see the fibre channel adapters under storage adapters of the host
3. The hosts that are not patched are still showing all the datastores
4. Host hardware is incompatible with the ESXi version.
Environment
VMware vSphere ESXi
Cause
1. HBA driver & firmware mismatch or other HBA related hardware problems.
2. HBA model is incompatible with the ESXi version
Resolution
WORKING HOST
[root@NotPatched:~] hostname; vmware -vl
ESXi-1
VMware ESXi 7.0.3 build-20328353
VMware ESXi 7.0 Update 3
Working host is detecting the Emulex HBA
[root@rri2esxpl415:~] esxcli storage san fc list
Adapter: vmhba3
Port ID: 000000
Node Name: 20:#:00:00:#:00:00:#
Port Name: 10:00:00:#:00:00:00:#
Speed: 0 Gbps
Port Type: Unknown
Port State: LINK DOWN
Model Description: Emulex LightPulse LPe16002B-M6-D 2-Port 16Gb Fibre Channel Adapter
Hardware Version: 0000000d
OptionROM Version: 11.0.270.2002
Firmware Version: 11.0.270.2002
Driver Name: lpfc
DriverVersion: 14.0.169.26
Adapter: vmhba4
Port ID: 000000
Node Name: 20:#:00:00:#:00:00:#
Port Name: 10:00:00:#:00:00:00:#
Speed: 0 Gbps
Port Type: Unknown
Port State: LINK DOWN
Model Description: Emulex LightPulse LPe16002B-M6-D 2-Port 16Gb Fibre Channel Adapter
Hardware Version: 0000000d
OptionROM Version: 11.0.270.2002
Firmware Version: 11.0.270.2002
Driver Name: lpfc
DriverVersion: 14.0.169.26
Adapter: vmhba5
Port ID: 020660
Node Name: 20:#:00:00:#:00:00:#
Port Name: 10:00:00:#:00:00:00:#
Speed: 16 Gbps
Port Type: NPort
Port State: ONLINE
Model Description: Emulex LightPulse LPe16002B-M6-D 2-Port 16Gb Fibre Channel Adapter
Hardware Version: 0000000d
OptionROM Version: 11.2.216.9
Firmware Version: 11.2.216.9
Driver Name: lpfc
DriverVersion: 14.0.169.26
Adapter: vmhba6
Port ID: 000000
Node Name: 20:#:00:00:#:00:00:#
Port Name: 10:00:00:#:00:00:00:#
Speed: 0 Gbps
Port Type: Unknown
Port State: LINK DOWN
Model Description: Emulex LightPulse LPe16002B-M6-D 2-Port 16Gb Fibre Channel Adapter
Hardware Version: 0000000d
OptionROM Version: 11.2.216.9
Firmware Version: 11.2.216.9
Driver Name: lpfc
DriverVersion: 14.0.169.26
[root@rri2esxpl415:~] esxcli software vib list | grep -i lpfc
lpfc 14.0.169.26-5vmw.703.0.50.20036589 VMW VMwareCertified 2023-08-10
NON-WORKING HOST
[root@Patched:~] hostname; vmware -vl
ESXi-2
VMware ESXi 7.0.3 build-23307199
VMware ESXi 7.0 Update 3
This host is not detecting the Emulex HBA
[root@rri2esxpl414:~] esxcli storage san fc list
[root@rri2esxpl414:~] esxcli software vib list | grep -i lpfc
lpfc 14.2.567.0-1OEM.700.1.0.15843807 EMU VMwareCertified 2024-08-17
NOTE: The patch applied to the host has updated the driver from 14.0.169.26 to 14.2.567.0.
[root@rri2esxpl415:~] vmkchdev -l | grep -i vmhba
0000:00:11.4 8086:8d62 1028:0601 vmkernel vmhba1
0000:00:1f.2 8086:8d02 1028:0601 vmkernel vmhba2
0000:02:00.0 1000:005d 1028:1f49 vmkernel vmhba0
0000:03:00.0 10df:e200 10df:e206 vmkernel vmhba5
0000:03:00.1 10df:e200 10df:e206 vmkernel vmhba6
0000:81:00.0 10df:e200 10df:e206 vmkernel vmhba3
0000:81:00.1 10df:e200 10df:e206 vmkernel vmhba4
Hardware manufacturers like QLogic, Emulex, Brocade & Broadcom to name a few test their products (NIC/HBA/CNA) with specific versions of driver and firmware that is compatible with each other. When you are patching, updating or upgrading your ESXi host, you may obliviously ignore to think about this vital piece of information and later hit a roadblock after the upgrade is complete.
Drivers is a software that enables your hardware devices to communicate with your operating system and applications. If your drivers are not compatible with your current system, you might experience errors, crashes, or reduced functionality.
Firmware is a software that controls the basic functions of your hardware devices, such as booting, loading & other operations.
If your driver or firmware is incompatible or outdated, you may encounter problems with the hardware or within the operating system (ESXi). And because these devices connect to your network or storage they can have a significant performance impact on your virtual machines overall performance. Therefore, its very critical to check the below information before patching your hosts.
1. Host hardware compatibility with ESXi version. Contact your server vendor or look up this information on their website.
2. Driver & firmware compatibility with ESXi version. Look up VMware Compatibility Guide to find this information.
In relevance to this KB, HBAs can go missing when driver and firmware versions become incompatible or the HBA hardware is dead. Please contact your server vendor to download and install the latest HBA firmware version compatible with the driver in alignment with the VCG.
It's also harmless to upgrade to the highest version of firmware available for your adapters, as long as you are willing to apply this change to a single server and monitor it for a few days before applying it to the cluster.
CAUTION: Flashing the BIOS on a HBA can sometimes lead to unknown hardware problems with the adapter if it's very old or even require a replacement if it kills the hardware. Please proceed to do this update under the supervision of your server manufacturer to be on the safer side.
Additional Information
Determining Network/Storage firmware and driver version in ESXi
For older driver refer
https://knowledge.broadcom.com/external/article/318054/after-upgrading-the-esxi-host-to-703-una.html
Feedback
