ESXi Upgrade Fails with Failed to Query File System Stats Due to Fibre Channel Frame Drops

Products

• VMware vSphere ESXi

Issue/Introduction

When you're upgrading a VMware ESXi 8.0 host using ESXi Lifecycle Manager (vLCM) or the Command Line Interface (CLI), the remediation process fails at approximately 32%.  You see the following error message observed in the task console or logs: (this may occur specifically when upgrading from ESXi version 8.0u3i to 8.0u3k.)

• Failed to query file system stats: "Errors: Cannot open volume"

Review of the vmkernel.log reveals frequent Fibre Channel (FC) instability and frame drop events occurring during the upgrade attempt:

• WARNING: iodm: vmk_IodmEvent:###: vmhba#: FRAME DROP event has been observed ### times in the last one minute. This suggests a problem with Fibre Channel link/switch!.

Environment

• Product: VMware ESXi
• Version: 8.0.x

Cause

The ESXi upgrade process requires stable communication with all presented storage volumes. High frequencies of Fibre Channel "FRAME DROP" events on a specific Host Bus Adapter (HBA) cause pathing instability and latency. This prevents the ESXi host from successfully querying file system statistics for mounted volumes, causing the upgrade to abort.

Resolution

To resolve this issue, the underlying storage path instability must be addressed before reattempting the upgrade: (Also reference: ESXi host upgrade fails with the error message "Failed to query file system stats: Errors:)

1. You Identify the Affected Path: Review host logs to confirm the specific HBA (e.g., vmhba3) reporting the frame drops.
2. Inspect Physical Infrastructure: Coordinate with the storage and networking teams to inspect the physical Fibre Channel links, SFP modules, cables, and switch ports associated with the affected HBA.
3. Verify Storage Visibility: Once the physical layer is stabilized, log in to the ESXi host via SSH and run the following command to ensure all datastores are visible and mounted: esxcli storage filesystem list
4. Refresh Storage View: If any datastores appear stale or inaccessible, refresh the host’s storage view by running: esxcli storage core adapter rescan -a vmkfstools -V
5. Retry Upgrade: Reinitiate the ESXi host upgrade via vLCM or CLI once storage connectivity is stable.

Additional Information

Frequent frame drops are typically indicative of hardware degradation or configuration mismatches in the Fabric (e.g., faulty SFPs or mismatched speed/duplex settings). Even if the host remains operational, the increased metadata overhead during an upgrade often triggers a failure when these pathing errors are present.
ESXi host upgrade fails with the error message "Failed to query file system stats: Errors:"