NVRAM destage failure predicted for device: NVRAM12 errors

Applies to

• NVRAM12 module
• AFF A1K
• AFF A90
• AFF A70
• AFF C80
• AFX 1K
• ASA A1K
• ASA A90
• ASA A70
• FAS90
• FAS70

Issue

• In ONTAP Autosupport section  EMS-LOG-FILE.GZ shows weekly:

env_mgr: nv.flash.predict.fail:alert]: NVRAM destage failure predicted for device: NVRAM12.

• In ONTAP Autosupport section SP-LATEST-CONSOLE-LOGS shows frequent:

[nvram.hw.initFail:CRITICAL]: NVRAM hardware initialization failed: Failed to read SSD smart log.

• In some cases a node may fail to boot,  as well as Pause ANDU  (Automatic Non Disruptive Upgrade) due to missing NVRAM. In SP-LATEST-CONSOLE-LOGS:

Starting netapp_loadnvram - 09:55:12
Command info failed.
Ending netapp_loadnvram - 09:55:12
WARNING: Unable to get sysid!
...
nv.flash.unable.to.monitor:ALERT]: Unable to monitor NVRAM device "NVRAM12".
...
nv.none:EMERGENCY]: No NV device has been detected.
...
System halting... 

• In some cases fru-check command on the NVRAM will show: 

::*> system fru-check show -node * -fru-name NVRAM
Node               FRU                          Status
------------------ ---------------------------- -----------
node-1
                   NVRAM12                      pass
node-2
                   NVRAM12                      unknown
2 entries were displayed.

• The SSD Destage device shows faulted:

cluster1::> system node environment sensors show -state !normal
Node Sensor                 State Value/Units Crit-Low Warn-Low Warn-Hi Crit-Hi
---- --------------------- ------ ----------- -------- -------- ------- -------
cluster1-01
     SSD Mode              failed
     SSD Status            failed
     SSD Destg Status       fault
                                          BAD