Register State Change Notifications (RSCNs) messages from the HBA driver observed in /var/log/vmkernel.log on ESXi
Article ID: 392902
Updated On:
Products
Issue/Introduction
Depending on the HBA driver, the following events would be visible in /var/log/vmkernel.log when a Registered State Change Notification (RSCN) has been received from a device in the same fabric zone:
Cisco NFNIC
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu179:2099218)nfnic: <2>: INFO: fdls_process_rscn: 3918: RSCN payload_len: 0x800 page_len: 0x4
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu179:2099218)nfnic: <2>: INFO: fdls_process_rscn: 3927: RSCN for port id: 0x20040
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu179:2099218)nfnic: <2>: INFO: fdls_process_rscn: 3927: RSCN for port id: 0x20060
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu179:2099218)nfnic: <2>: INFO: fdls_process_rscn: 3937: RSCN received for num_ports: 0 payload_len: 2048 page_len: 4
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu179:2099218)nfnic: <2>: INFO: fdls_send_gpn_ft: 1272: Sending fabric GPNFT for fcid:0x302d2
Qlogic (qlnativefc)
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): Asynchronous RSCN UPDATE.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): RSCN database changed -- 0003 0500 0000.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): Asynchronous RSCN UPDATE.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): RSCN database changed -- 0003 0700 0000.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): Asynchronous RSCN UPDATE.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): RSCN database changed -- 0003 0c00 0000.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): Asynchronous RSCN UPDATE.
YYYY-MM-DDTHH:MM:SS.SSSSZ cpu10:3591802)qlnativefc: vmhba2(6:0.0): RSCN database changed -- 0003 0e00 0000.
Emulex (lpfc)
YYYY-MM-DDTHH:MM:SS.SSSSZ In(182) vmkernel: cpu20:2098304)lpfc: lpfc_els_rcv_rscn:7907: vmhba0 0214 RSCN received Data: x800220 x0 x4 x1
YYYY-MM-DDTHH:MM:SS.SSSSZ In(182) vmkernel: cpu20:2098304)lpfc: lpfc_els_rcv_rscn:7914: vmhba0 5973 RSCN received event x0 : Address format x02 : DID x230000
YYYY-MM-DDTHH:MM:SS.SSSSZ In(182) vmkernel: cpu114:2098352)lpfc: lpfc_els_rcv_rscn:7907: vmhba1 0214 RSCN received Data: x800220 x0 x4 x1
YYYY-MM-DDTHH:MM:SS.SSSSZ In(182) vmkernel: cpu114:2098352)lpfc: lpfc_els_rcv_rscn:7914: vmhba1 5973 RSCN received event x0 : Address format x02 : DID x240000
Environment
VMware ESXi 6.x
VMware ESXi 7.x
VMware ESXi 8.x
Cause
A Registered State Change Notification (RSCN) is a notification frame that is sent to devices that are zoned together and are registered to receive a State Change Notification (SCN). The RSCN is responsible for notifying all devices of fabric changes. The following general list of actions can cause an RSCN to be sent through your fabric:
- A new device has been added to the fabric
- An existing device has been removed from the fabric
- A zone has changed
- A switch name or an IP address has changed
- Nodes have left or joined the fabric, such as zoning, powering on or shutting down a device, or zoning changes.
For more information, please see the following documentation: Registered State Change Notifications
Resolution
The events being present in the /var/log/vmkernel.log are an indication that something has changed in the fabric, usually a device rebooting or a flakey port causing a device to drop from the fabric and to reconnect.
Since the industry standard is to use Single Initiator zoning, the device rebooting or dropping/reconnecting would always be an array target port. Since the RSCN event typically requires the HBA(s) receiving the RSCN to drop from the fabric and reconnect, this would cause storage paths to go dead/offline temporarily while the HBA(s) reconnect to the fabric.
RSCNs are disruptive to the storage IO path so if RSCNs are observed when no storage array target ports were intentionally rebooted then there is a high probability of there being a bad SFP or low light levels on the array side fabric port causing the port itself to drop from the fabric temporarily.
Feedback
