StorageGRID appliance kernel panic after Correctable ECC Error Logging Limit Reached

• StorageGRID detects Unable to communicate with node alert due to storage node reboot
• storagegrid_crash_dmesg.<TIMESTAMP>.log shows Kernel panic after many I/O errors

[61824251.190163] sd 17:0:0:240: rejecting I/O to offline device
[61824251.193978] sd 17:0:0:238: rejecting I/O to offline device
[61824251.193982] sd 17:0:0:246: rejecting I/O to offline device
[61824251.193984] sd 17:0:0:244: rejecting I/O to offline device
[61824251.193985] sd 17:0:0:242: rejecting I/O to offline device
[61824251.193994] device-mapper: multipath: Failing path 68:0.
[61824251.193996] device-mapper: multipath: Failing path 67:224.
[61824251.194005] device-mapper: multipath: Failing path 67:192.
[61824251.194039] sd 6:0:0:242: rejecting I/O to offline device
[61824251.194043] device-mapper: multipath: Failing path 8:96.
[61824251.194056] sd 15:0:0:242: rejecting I/O to offline device
[61824251.194058] device-mapper: multipath: Failing path 65:128.
[61824251.194078] XFS (dm-11): metadata I/O error in "xlog_iodone" at daddr 0x27c26a78 len 64 error 5
[61824251.194080] XFS (dm-11): xfs_do_force_shutdown(0x2) called from line 1271 of file fs/xfs/xfs_log.c.  Return address = 00000000557f8fa2
[61824251.194140] XFS (dm-11): Log I/O Error Detected.  Shutting down filesystem
[61824251.194140] XFS (dm-11): Please umount the filesystem and rectify the problem(s)
[61824251.194151] sd 6:0:0:244: rejecting I/O to offline device
[61824251.194153] device-mapper: multipath: Failing path 8:128.
[61824251.194162] sd 15:0:0:244: rejecting I/O to offline device
[61824251.194164] device-mapper: multipath: Failing path 65:160.
[61824251.194187] XFS (dm-13): metadata I/O error in "xlog_iodone" at daddr 0x27c39778 len 64 error 5
[61824251.194191] XFS (dm-13): xfs_do_force_shutdown(0x2) called from line 1271 of file fs/xfs/xfs_log.c.  Return address = 00000000557f8fa2
[61824251.194193] sd 6:0:0:246: rejecting I/O to offline device
[61824251.194195] device-mapper: multipath: Failing path 8:160.
[61824251.194202] sd 15:0:0:246: rejecting I/O to offline device
[61824251.194203] device-mapper: multipath: Failing path 65:192.
[61824251.194216] XFS (dm-15): metadata I/O error in "xlog_iodone" at daddr 0x27c06fa8 len 64 error 5
[61824251.194218] XFS (dm-15): xfs_do_force_shutdown(0x2) called from line 1271 of file fs/xfs/xfs_log.c.  Return address = 00000000557f8fa2
[61824251.194242] XFS (dm-15): Log I/O Error Detected.  Shutting down filesystem
[61824251.194243] XFS (dm-15): Please umount the filesystem and rectify the problem(s)
[61824251.194245] XFS (dm-13): Log I/O Error Detected.  Shutting down filesystem
[61824251.194246] XFS (dm-13): Please umount the filesystem and rectify the problem(s)
[61824251.199608] sd 15:0:0:251: rejecting I/O to offline device
[61824251.205849] device-mapper: multipath: Failing path 67:128.
[61824251.212037] sd 15:0:0:251: rejecting I/O to offline device
[61824251.222021] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

• SEL_TEXT.txt downloaded by BMC Logs & Reports > IPMI Event Log > Download Event Logs by selecting Text File Type shows Correctable ECC error after Logging Limit Reached

Event ID        Time Stamp                    Severity    Sensor Name            Sensor Type                Description
    46        <TIMESTAMP>    [Information]    [Memory Error]    [Memory]        Correctable ECC(CPU0_E0) - Asserted
    45        <TIMESTAMP>    [Information]    [Memory Error]    [Memory]        Correctable ECC(CPU0_E0) - Asserted
    44        <TIMESTAMP>        [Information]    [Memory Error Dis]    [Event Logging Disabled]    Correctable Memory Error Logging Disabled - Asserted
    43        <TIMESTAMP>    [Critical]        [Memory Error]    [Memory]        Correctable ECC Error Logging Limit Reached(CPU0_E0) - Asserted​​​​​