Skip to main content
NetApp Knowledge Base

VMs on FC datastore experience continuous/frequent disconnections along with fcp.io.status: AEN 0x8048 events due to faulty storage SFP

  1. Last updated
  2. Save as PDF
Views:
1,805
Visibility:
Public
Votes:
1
Category:
fas-systems
Specialty:
san
Last Updated:

​ ​

Applies to

  • All NetApp FAS/AFF Hardware Platforms
  • All Data ONTAP Versions
  • FC Protocol

Issue

  • Virtual machines on datastore connected experience continuous/Frequent disconnection, slow response, freezing and crashing on a single node.
  • This can also happen on non-virtual infrastructure with FC volumes.
  • Moving the volumes to another node solves the problem.
  • AEN 0x8048 (RECV_ERROR) messages frequently detected in event logs for the affected node, in specific FCP adapter.

Thu Mar 05 21:20:10 CET [Cluster1-01: fct_tpd_work_thread_0: fcp.io.status:debug]: STIO Adapter:0f AEN 0x8048 (RECV_ERROR) MboxStatus1 0x200 MboxStatus2 0xc0

  • Affected FCP adapters might have less "received optical power" than the rest on the same node. Check this with the command:

network fcp adapter show -node Cluster1-01 -adapter * -fields sfp-rx-power

  • Below port login events can be seen in EMS logs.

[?] Wed Sep 04 00:29:20 +0000 [xxxx: fct_tpd_work_thread_0:scsitarget.fct.portLogin:notice]: Login at target FC port: '0f' by initiator port: 10:00:9e:85:xx:xx:00:xx' address 0x11xxx. The target virtual port is:'NetApp FC Target Port (8324) Xxxx:xxxx_1x_0f'.
[?] Wed Sep 04 00:32:21 +0000 [xxxx: fct_tpd_work_thread_0:scsitarget.fct.portLogin:notice]: Login at target FC port: '0f' by initiator port: '10:00:9e:85:xx:xx:00:xx' address 0x11xxx. The target virtual port is:'NetApp FC Target Port (8324) XGxxx3:xxx_1x_0f'.

  • No internal latency detected for the storage system. 
  • Below troubleshooting steps can be followed before proceeding with solution.
    • Verify SAN Fabric is clean of problems mentioned below:
      • Bad Start of frame
      • Bad End of frame
      • CRC error
      • Disparity error
      • Length error
      • Link Reset (LR) primitive sequence received
    • Check that the AEN 0X8048 messages are no longer present in the event logs.
    • Verify that the "received optical power" for all the FC adapters is within expected values. Check: What should the RX light power of an SFP be?
    • Perform cable testing and see if there are any loosely connected cables.

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.