PSOD on Cisco servers. PF Exception 14 on nfnic module
search cancel

PSOD on Cisco servers. PF Exception 14 on nfnic module

book

Article ID: 412357

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • PSOD occurs on Hosts when nfnic 5.0.0.46-1OEM.803.0.0.24022510 or 5.0.0.48-1OEM.803.0.0.24022510 is in use



    VMware ESXi 8.0.3 [Releasebuild-24784735 x86_64]

#PF Exception 14 in world 2111325:fnic_world_f IP 0x42001657fccl addr 0x45b9422cd2d0
PTEs:0x3040071023:0x135a93063:0x0;

Module(s) involved in panic: [nfnic 5.0.0.46-10EM.803.0.0.24022510 (External)]
cr0=0x8001003d cr2=0x45b9422cd2d0 cr3=0x6c867000 cr4=0x14216c
FMS=06/55/7 uCode=0x5003801
frame=0x453928f9bdb0 ip=0x42001657fccl err=0x0 rflags=0x10283
rax=0xfffff6894d5a1557 rbx=0x4311d80013c0 rcx=0x35
rdx=0x763 rbp=0x4311e4a2b0f8 rsi=0x0
rdi=0x98360931a6c r8=0x4311e4a2b0f8 r9=0x7b8eed
r10=0x3fc00000 г11=0x420047401040 г12=0x405
r13=0x0 r14=0x4200165abc28 r15=0x45b9422cd280
*PCPU9:2111325/fnic_world_func-1-0x4311d8079ba
PCPU 0: SSSSSSSUSSUUVSSVVSSSVSSSSSVSSSVSSSUVUSSSUVSSVV
Code start: 0x420015400000 VMK uptine: 0:01:04:34.067
0x453928f9be70: [0x42001657fcc1]fnic_abort_cnd@(nfnic)#<None>+0x921 stack: 0x976b2e10792
0x453928f9bf00: [0x42001658041b]fnic_virt_reset_abort@(nfnic)#<None>+0x1d0 stack: 0x4311d8079ba0
0x453928f9bfa0: [0x42001559fae4]vmkWor 1dFunc@vmkernel#nover+0x31 stack: 0x42001559fae0
0x453928f9bfe0: [0x420015adc88e ]CpuSched_StartWor1d@vnkernel#nover+0xbf stack: 0x0
0x453928f9c000: [0x420015544faf ]Debug_IsInitialized@vnkernel#nover0xc stack: 0x0
base fs=0x0 gs=0x420042400000 Kgs=0x0
No disk partition configured to dump data.
Finalized dump header (18/18) FileDump: Successful.
No port for remote debugger. "Escape" for local debugger.

  • PSOD on 5.0.0.48-1OEM.803.0.0.24022510
    VMware ESXi 8.0.3 [Releasebuild-25429389 x86_64]
    #PF Exception 14 in world 8172737:fnic_world_f IP 0x4200159639c0 addr 0x45da0bc001e8
    PTEs:0x8000086023;0x80b8044063:0x0;
    
    Module(s) involved in panic: [nfnic 5.0.0.48-1OEM.803.0.0.24022510 (External)]
    cr0=0x8001003d cr2=0x45da0bc001e8 cr3=0x610b5000 cr4=0x14216c
    FMS=06/55/4 uCode=0x2007006
    frame=0x45399f31bda0 ip=0x4200159639c0 err=0x0 rflags=0x10246
    rax=0x0 rbx=0x43138c2013c0 rcx=0x401
    rdx=0x21 rbp=0x45da0bc001b0 rsi=0x4200480016c0
    rdi=0xffffffffffffffff r8=0x420048000000 r9=0x0
    10=0x0 r11=0x420048001040 r12=0x43138c201248
    13=0x42001598fde8 r14=0x16f r15=0x47
    *PCPU32:8172737/fnic_world_func-1-0x43138c2561e
    PCPU 0: VSSUUUIUSUUSUVVSVSIVSVVVIVSVVUVUSUSSSUSSUUSSSUSUUUIIIIUSUSUIUUVU
    PCPU 64: VSVUVUSUUSUIUSUS
    Code start: 0x420014800000 VMK uptime: 62:20:03:31.218
    0x45399f31be60:[0x4200159639c0]fnic_abort_cmd@(nfnic)#<None>+0xa80 stack: 0x273
    0x45399f31bf00:[0x420015963fdb]fnic_virt_reset_abort@(nfnic)#<None>+0x1d0 stack: 0x43138c2561e0
    0x45399f31bfa0:[0x4200149a00fc]vmkWorldFunc@vmkernel#nover+0x31 stack: 0x4200149a00f8
    0x45399f31bfe0:[0x420014edddfe ]CpuSched_StartWorld@vmkernel#nover+0xbf stack: 0x0
    Bx45399f31c000:[0x4200149455af ]Debug_IsInitialized@vmkernel#nover+0xc stack: 0x0
    base fs=0x0 gs=0x420048000000 Kgs=0x0
    YYYY-MM-DDTHH:MM:SS.MSZ cpu4:2097840)Jumpstart plugin restore-configuration-files activation failed.
    No disk partition conf igured to dump data.
    Finalized dump header (18/18) FileDump: Successful.
    No port for remote debugger. "Escape" for local debugger.
  • VMkernel logs (/var/run/log/vmkernel) may have multiple entries listed below, suggesting storage IO failures confirming the scenario. 

nfnic: <1>: INFO: fnic_terminate_ioreq: 2971: io_req: 0x########## sc: 0x########## tag: 0xcd CMD_FLAGS: 0x1000 Found IO in FNIC_IOREQ_CMD_PENDING state on lun

Environment

  • VMware vSphere ESXi 8.x
  • VMware vSphere ESX 9.x

Cause

This issue is documented/reported by the vendor Cisco and is currently under investigation by their team. 

Cisco Bug: CSCwq55757

Resolution

Contact the hardware vendor for further assistance, potential workarounds, and their official recommendation for a stable alternative to nfnic 5.0.0.46 or nfnic 5.0.0.48

Additional Information

Uninstalling the driver Instructions for uninstalling third-party VIBs from an ESXi host.

Installing a more stable version via VIB - Patching ESXi host using Command Line