Transient storage path failures and "fnic_fdls_link_status_change link up:0" with Cisco UCS VIC adapters
search cancel

Transient storage path failures and "fnic_fdls_link_status_change link up:0" with Cisco UCS VIC adapters

book

Article ID: 432676

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

VMware ESXi hosts using Cisco UCS Virtual Interface Cards (VIC) with the nfnic driver experience sudden, transient storage path drops. Multiple hosts may report the failure simultaneously.

To verify if this issue is occurring, log in to the affected ESXi host via SSH (using a client such as PuTTY) and review the /var/run/log/vmkernel.log.

Symptoms observed in the vmkernel.log include:

  • Physical Link Drop: An abrupt physical link drop on the adapter: In(182) vmkernel: cpu93:2098107)nfnic: <1>: INFO: fnic_fdls_link_status_change: 98: fnic1: FDLS link status change link up:0, usefip:0

  • Path Failures: The ESXi scsiCorrelator service marking paths as dead, and Native Multipathing Plugin (NMP) operations failing immediately following the drop: [scsiCorrelator] 17284358534073us: [vob.scsi.scsipath.pathstate.deadver2] scsiPath vmhba2:C0:T19:L1 changed state from on WARNING: VMW_SATP_ALUA: satp_alua_getTargetPortInfo:190: Could not get page 83 INQUIRY data for path "vmhba2:C0:T18:L1" - No connection

  • Physical Link Recovery: Physical link recovery occurring seconds later: In(182) vmkernel: cpu93:2098107)nfnic: <1>: INFO: fnic_fdls_link_status_change: 98: fnic1: FDLS link status change link up:1, usefip:0 In(182) vmkernel: cpu93:2098107)nfnic: <1>: INFO: fnic_handle_link: 1068: fnic1: link up

  • Array Response: During the recovery sequence, the storage array may return SCSI sense codes such as UNIT ATTENTION (Reported LUNs Data Has Changed) or ILLEGAL REQUEST as the host re-registers with the fabric.

Environment

VMware ESXi 8.0.x
Cisco UCS Virtual Interface Cards (VIC) nfnic driver

Cause

The ESXi host is operating as designed and reacting to a physical layer (Layer 1) disruption. The root cause is a brief physical link drop originating from the Cisco UCS Fabric Interconnect (FI), an upstream SAN switch, or a faulty Twinax/fiber cable. When the link drops, ESXi instantly fails over paths until the hardware link is restored.

Resolution

To prevent recurrence, investigate the physical hardware layer:

  1. Identify the exact timestamp of the link up:0 event in the ESXi vmkernel.log.

  2. Cross-reference this timestamp with the Cisco UCS Fabric Interconnect (FI) logs. Look for port flaps, IOM/FEX resets, or VIF down events.

  3. If multiple hosts report the link drop at the exact same millisecond, the fault lies in a shared upstream component (e.g., Fabric Interconnect reboot or switch-side port flap).

  4. If the drop is isolated to a single host, physically inspect the SFP transceivers and cabling connected to the affected Cisco VIC.

Additional Information

Engage Cisco Technical Assistance Center (TAC) if the Fabric Interconnect logs indicate unexpected port flaps or hardware resets.

Powered by Wolken Software