Correctable memory errors reporting in ONTAP versions with dynamic thresholds

Applies to

ONTAP versions:

• 9.1P18 and later P releases
• 9.3P11 and later P releases
• 9.4P6 and later P releases
• 9.5 and later major releases

Platforms:

• AFF A900 / FAS9500
• AFF A800 / AFF C800
• AFF A700s
• AFF A700 / FAS9000
• AFF A400 / AFF C400 / FAS8300 / FAS8700
• AFF A300 / FAS8200
• AFF A250 / AFF C250 / FAS500f
• AFF A220 / AFF C190 / FAS27x0
• AFF A200 / FAS26x0
• AFF80x0 / FAS80x0

• CriticalCECCCountMemErrAlert

Note: For all other ONTAP platforms and ONTAP versions see How to troubleshoot correctable memory errors on FAS and AFF systems

Issue

• Health Alert AutoSupport message:

HA Group Notification (Health Monitor process nphm: CriticalCECCCountMemErrAlert[DIMM-16]) ALERT

• EMS reports:

[Cluster-01: mgwd: callhome.hm.alert.critical:alert]: Call home for Health Monitor process nphm: CriticalCECCCountMemErrAlert[DIMM-16].

• Health Monitor reports

::> system health alert show
               Node: Node1
           Alert ID: CriticalCECCCountMemErrAlert
           Resource: DIMM-16
           Severity: Critical
    Indication Time: Wed Jan 10 16:45:18 2024
           Suppress: false
        Acknowledge: false
     Probable Cause: The DIMM has degraded, leading to memory errors.
  Possible Effect: Memory issues can lead to a catastrophic system panic,
                     which can lead to data downtime on the node.
 Corrective Actions: 1. Contact technical support to obtain a new DIMM of the same specification.
                     2. If possible, perform a takeover of this node and bring the node down for maintenance.
                     3. Refer to the DIMM replacement guide for your given hardware platform to replace the DIMM.
                     4. Bring the storage system online.

• Memory dimm show reports the dimm status as degraded

Cluster::> memory dimm show
              DIMM     UECC  CECC  Alert    CPU           Slot          Failure
Node          Name    Count Count Method Socket Channel Number Status    Reason
------------- ------- ----- ----- ------ ------ ------- ------ ------- --------
Cluster-01
              DIMM-1      0     0 bucket      1       0      0 ok          none
              DIMM-2      0     0 bucket      1       0      1 ok          none
              DIMM-3      0     0 bucket      1       1      0 ok          none
              DIMM-4      0     0 bucket      1       1      1 ok          none
              DIMM-5      0     0 bucket      1       3      1 ok          none
              DIMM-6      0     0 bucket      1       3      0 ok          none
              DIMM-7      0     0 bucket      1       2      1 ok          none
              DIMM-8      0     0 bucket      1       2      0 ok          none
              DIMM-9      0     0 bucket      0       2      0 ok          none
              DIMM-10     0     0 bucket      0       2      1 ok          none
              DIMM-11     0     0 bucket      0       3      0 ok          none
              DIMM-12     0     0 bucket      0       3      1 ok          none
              DIMM-13     0     0 bucket      0       1      1 ok          none
              DIMM-14     0     0 bucket      0       1      0 ok          none
              DIMM-15     0     0 bucket      0       0      1 ok          none
              DIMM-16     0     0 bucket      0       0      0 degraded    none