System panic with PCI Error NMI from device(s): Link down

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 1,420

Visibility:: Public

Votes:: 0

Category:: fas-systems

Specialty:: hw

Last Updated:

Applies to

FAS80X0
AFF A80X0
AFF A300
AFF A700

Issue

Storage controller triggers PCI NMI error and results in system reboot.
SP system log displays error:
- PANIC: PCI Error NMI from device(s):ErrSrcID(CorrSrc(0x5f20),UCorrSrc(0)), RPT(0,3,0):PLX PCIE 8764 switch in slot 6 on Controller, Br[8764](95,4,0) in slot 6: Link down. in process idle: cpu25
- PANIC: PCI Error NMI from device(s):ErrSrcID(CorrSrc(0),UCorrSrc(0x680)), PLX PCIE 8748 switch on Controller, Br[8748](6,16,0): Link down. in process idle on release 8.2.4
  - From the sysconfig -PCI command output, it is determined that the link was down between Motherboard and card in slot 2:
    - Br[8748](6,16,0): PLX PCIE 8748 switch on Controller LinkCap(MaxLkSp(3),MaxLkWd(8),ASPM(3),L0(5),L1(0),Port(0)) LinkStatus(LkSp(2),LkWd(4),DLAct), Dv[150e](9,0,0) in slot 2: Intel 1G NIC in slot 2 on Controller LinkCap(MaxLkSp(2),MaxLkWd(4),ASPM(3),L0(6),L1(1),Port(0)) LinkStatus(LkSp(2),LkWd(4),SClk), Dv[150e](9,0,1) in slot 2: Intel 1G NIC in slot 2 on Controller LinkCap(MaxLkSp(2),MaxLkWd(4),ASPM(3),L0(6),L1(1),Port(0)) LinkStatus(LkSp(2),LkWd(4),SClk), Dv[150e](9,0,2) in slot 2: Intel 1G NIC in slot 2 on Controller LinkCap(MaxLkSp(2),MaxLkWd(4),ASPM(3),L0(6),L1(1),Port(0)) LinkStatus(LkSp(2),LkWd(4),SClk), Dv[150e](9,0,3) in slot 2: Intel 1G NIC in slot 2 on Controller LinkCap(MaxLkSp(2),MaxLkWd(4),ASPM(3),L0(6),L1(1),Port(0)) LinkStatus(LkSp(2),LkWd(4),SClk),
  - sysconfig -ac shows details of the card in slot 2:
    - sysconfig: slot 2 OK: X1049C: PCI-E Quad 10/100/1000 Ethernet 82580(v3.29 and above
  - PANIC: PCI Error NMI from device(s):DMI(0,0,0),Br[8c10](0,28,0): Link down. in process idle on release 9.1P11
    - From the output of PCI-HIERARCHY.XML , it is determined that link was down in NIC port on motherboard:
      - Br[8c10](0,28,0): PCI Device 8086:8c10 on Controller LinkCap(MaxLkSp(2),MaxLkWd(4),ASPM(3),L0(3),L1(2),DLAct,Port(1)) LinkStatus(LkSp(2),LkWd(4),SClk,DLAct), 2 Dv[1563](16,0,0): Intel Dual 10G NIC on Controller LinkCap(MaxLkSp(2),MaxLkWd(8),ASPM(2),L0(5),L1(4),Port(0)) LinkStatus(LkSp(2),LkWd(4),SClk), 2 Dv[1563](16,0,1): Intel Dual 10G NIC on Controller LinkCap(MaxLkSp(2),MaxLkWd(8),ASPM(2),L0(5),L1(4),Port(0)) LinkStatus(LkSp(2),LkWd(4),SClk),
  - PANIC: PCI Error NMI from device(s):ErrSrcID(CorrSrc(0),UCorrSrc(0x8010)), RPT(128,2,0):Br[3c04](128,2,0): Link down, ErrSrcID(CorrSrc(0),UCorrSrc(0x8018)), RPT(128,3,0):Br[3c08](128,3,0): Link down. in process idle on release 8.2.5P5 on Wed Jun 1 18:40:17 KST 2022
    - From the output of PCI-HIERARCHY.XML , it is determined that link was down in between Motherboard and IOXM
      - Br[3c08](128,3,0): PCI Device 8086:3c08 on Controller LinkCap(MaxLkSp(3),MaxLkWd(16),ASPM(0),L0(7),L1(0),Port(0)) LinkStatus(LkSp(3),LkWd(16),SClk,DLAct), Br[8732](145,0,0): PLX PCIE 8732 switch on IO Expansion LinkCap(MaxLkSp(3),MaxLkWd(16),ASPM(2),L0(6),L1(0),Port(0)) LinkStatus(LkSp(3),LkWd(16)), Br[8732](146,8,0): PLX PCIE 8732 switch on IO Expansion LinkCap(MaxLkSp(3),MaxLkWd(16),ASPM(3),L0(6),L1(0),Port(0)) LinkStatus(LkSp(3),LkWd(16),DLAct), Br[8748](147,0,0): PLX PCIE 8748 switch on IO Expansion LinkCap(MaxLkSp(3),MaxLkWd(16),ASPM(2),L0(6),L1(0),Port(0)) LinkStatus(LkSp(3),LkWd(16)), Br[8748](148,8,0): PLX PCIE 8748 switch on IO Expansion LinkCap(MaxLkSp(3),MaxLkWd(8),ASPM(3),L0(5),L1(0),Port(0)) LinkStatus(LkSp(2),LkWd(8),DLAct), Dv[10fb](149,0,0) in slot 9: Intel 10G NIC in slot 9 on IO Expansion LinkCap(MaxLkSp(2),MaxLkWd(8),ASPM(1),L0(4),L1(1),Port(0)) LinkStatus(LkSp(2),LkWd(8),SClk), Dv[10fb](149,0,1) in slot 9: Intel 10G NIC in slot 9 on IO Expansion LinkCap(MaxLkSp(2),MaxLkWd(8),ASPM(1),L0(4),L1(1),Port(0)) LinkStatus(LkSp(2),LkWd(8),SClk),