Corruption May Occur On PURE Storage Arrays When Connected via Cisco's NFNIC
search cancel

Corruption May Occur On PURE Storage Arrays When Connected via Cisco's NFNIC

book

Article ID: 318525

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

The following NFNIC messages are seen in the vmkernel.log:

WARNING: nfnic: <4>: fnic_fcpio_icmnd_cmpl_handler: 1636: sc: 0x459b12896d00 tag: 0x4e1 hdr status: FCPIO_ABORTED IO failure!
WARNING: nfnic: <4>: fnic_fcpio_icmnd_cmpl_handler: 1636: sc: 0x459b128ea7c0 tag: 0x507 hdr status: FCPIO_ABORTED IO failure!
WARNING: nfnic: <4>: fnic_fcpio_icmnd_cmpl_handler: 1636: sc: 0x459b1280c880 tag: 0x533 hdr status: FCPIO_ABORTED IO failure!


This may also lead to VMFS Metadata On-Disk corruption and/or corruption within the GuestOS of VMs

CTIO errors or Out of Order frames may be apparent in the SAN. Read/Writes may fail, but be marked as successful and can cause data inconsistency on remote storage datastores.

Environment

VMware ESXi 6.7.x
VMware vSphere ESXi 6.7

Cause

The Cisco UCS NFNIC  driver allows aborted Fibre Channel Sequence aborted by the target to be marked as successful instead of failed. This will lead to data inconsistency as writes fail due to SAN Out of Order frames and CTIO timeouts are not retried by the host.

This issue affects customer environments with ESXi 6.7 and later running Cisco UCS NATIVE FNIC driver (NFNIC). 

The Purity versions affected are as follows: 5.3.6 to 5.3.14, 6.0.0 to 6.0.5, and 6.1.0 to 6.1.2

Resolution

Cisco has released an updated Native FNIC driver (4.0.0.65) that addresses the FCP abort issue and prevents the lost write situation that leads to corruption.

For more information reference the CISCO defect CSCvx64370


Workaround:

If the environment meets the above criteria, until the long term fix is available from Cisco, we recommend upgrading as soon as possible to maintain data integrity. The issue can be mitigated by upgrading to Purity version 5.3.15+, 6.0.6+, and 6.1.3+.

Pure Storage Support will be happy to assist customers with this upgrade.