System down due to a dimm_uecc_error
Applies to
- FAS2820
- ONTAP 9.13.1
Issue
System down due to a dimm_uecc_error
EMS
<LR d="24Jan2025 10:50:28" n="node-02" t="xxxxxxxxxx" id="xxxxxxxxxx/xxxxxx" p="5" s="Ok" o="cf_hwassist" vf="" type="0" seq="328428" supp="9" ><cf_hwassist_takeoverTrapRecv_1 partnerName="node-01" trap_type="system_down" trap_event="dimm_uecc_error"/></LR><LR d="24Jan2025 10:50:28" n="node-02" t="xxxxxxxxxx" id="xxxxxxxxxx/xxxxxx" p="5" s="Ok" o="cf_hwassist" vf="" type="0" seq="328429" ><cf_hwassist_takeoverTrapRecv_1 partnerName="node-01" trap_type="system_down" trap_event="power_cycle_via_sp"/>BMC log(system log)
Symptom 1
================ Log #2 start time Sun Jan 19 21:40:27 2025BIOS Version: 19.2Portions Copyright (C) 2014-2023 NetApp, Inc. All Rights Reserved.Initializing System Memory ...HALT! Memory Initialize Fatal Error.Error Action: HALT System.================ Log #6 start time Mon Jan 20 12:46:13 2025BIOS Version: 19.2Portions Copyright (C) 2014-2023 NetApp, Inc. All Rights Reserved.Initializing System Memory ...HALT! Memory Initialize Fatal Error.Error Action: HALT System.================ Log #8 start time Mon Jan 20 20:37:46 2025BIOS Version: 19.2Portions Copyright (C) 2014-2023 NetApp, Inc. All Rights Reserved.Initializing System Memory ...HALT! Memory Initialize Fatal Error.Symptom 2
Local System ID: xxxxxxxxxPress Ctrl-C for Maintenance menu to release disks.Jan 24 10:50:27 [node-01:cf.fmns.skipped.disk:notice]: While releasing the reservations in "Waiting For Giveback" state Failover Monitor Node State(fmns) module skipped the disk 0b.00.1 that is owned by xxxxxxxxx and reserved by xxxxxxxxx.Boot argument NfsRdmaOk not specified. Default (set) is in effect.Boot argument NfsIpOk not specified. Default (set) is in effect.FWU 2nd trigger pointFWU has no post firmware update action registered.pnso provider init started.Notice : pnso card not detected.hwo: Node is using hardware provider : 1.ECC error at DIMM-1: F47A2F80,ADDR 0x5a82d34c0,(Node(0), Memory controller(0), CH(0), DIMM(0), Rank(0), Bank Group(3), Bank(0x2), Row(0x9505), Col(0x350)) Uncorrectable Machine Check Error at CPU2. ICL_IMC0_C0 Error: STATUS<0xbe00004200a00090>(VALID,UC,EN,MISCV,ADDRV,PCC,CORR_ERR_STAT(0),CORR_ERR_CNT(0x1),OTHER_INFO(0x2),MSCOD(0xa0),MCACOD(0x90)),MISC<0x08000b04a829a886>(EXTRA_ERR_INFO(0x40005825414d4),ADDR_MODE(0x2),REC_ERR_LSB(0x6)),ADDR<0x00000005a82d34c0>(ADDRESS(0x5a82d34c0))Node(0), Memory controller(0), CH(0), DIMM(0), Rank(0), Bank Group(3), Bank(0x2), Row(0x9505), Col(0x350), Device(0), DQ-Burst(3,0),Requesting SP to power cycle the filer to attempt to clear the Machine Check Event…ECC error at DIMM-1: F47A2F80,ADDR……ECC error at DIMM-1: F47A2F80,ADDR……ECC error at DIMM-1: F47A2F80,ADDR……ECC error at DIMM-1: F47A2F80,ADDR……Local System ID: xxxxxxxxxPress Ctrl-C for Maintenance menu to release disks.Jan 24 11:14:28 [node-01:cf.fmns.skipped.disk:notice]: While releasing the reservations in "Waiting For Giveback" state Failover Monitor Node State(fmns) module skipped the disk 0b.00.1 that is owned by xxxxxxxxx and reserved by xxxxxxxxx.Boot argument NfsRdmaOk not specified. Default (set) is in effect.Boot argument NfsIpOk not specified. Default (set) is in effect.FWU 2nd trigger pointFWU has no post firmware update action registered.pnso provider init started.Notice : pnso card not detected.hwo: Node is using hardware provider : 1.ECC error at DIMM-1: F47A2F80,ADDR 0x5a82d36c0,(Node(0), Memory controller(0), CH(0), DIMM(0), Rank(0), Bank Group(3), Bank(0x2), Row(0x9505), Col(0x370)) Uncorrectable Machine Check Error at CPU0. ICL_IMC0_C0 Error: STATUS<0xfe00004200a00090>(VALID,OVER,UC,EN,MISCV,ADDRV,PCC,CORR_ERR_STAT(0),CORR_ERR_CNT(0x1),OTHER_INFO(0x2),MSCOD(0xa0),MCACOD(0x90)),MISC<0x08000b04a829b886>(EXTRA_ERR_INFO(0x40005825414dc),ADDR_MODE(0x2),REC_ERR_LSB(0x6)),ADDR<0x00000005a82d36c0>(ADDRESS(0x5a82d36c0))Node(0), Memory controller(0), CH(0), DIMM(0), Rank(0), Bank Group(3), Bank(0x2), Row(0x9505), Col(0x370), Device(0), DQ-Burst(3,4),Requesting SP to power cycle the filer to attempt to clear the Machine Check Event