Servers cannot write to the LUN's due to WAFL very low on memory
Applies to
Issue
- Node is offline (hung) with aggregates state unknown and not responding to the ONTAP CLI:
::> aggr show
Info: Node node_name_2 that hosts aggregate aggr0_node_name_2 is offline
Node node_name_2 that hosts aggregate aggr2_node_name_2 is offline
Aggregate Size Available Used% State #Vols Nodes RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
...
aggr0_node_name_2 - - - unknown - node_name -
aggr2_node_name_2 - - - unknown - node_name -
::> storage failover show
Takeover
Node Partner Possible State Description
-------------- -------------- -------- -------------------------------------
node_name_1 node_name_2 true Connected to node_name
node_name_2 node_name_1 - Up. Node accessible via HA-IC, but
cluster access failed
2 entries were displayed.
- Servers cannot write on LUN's and data and root aggregates are seen offline. EMS logs example:
Sat Jan 09 04:14:35 CET [node_name-01: scsit_lu_1: wafl.memory.statusVeryLowMemory:alert]: WAFL is running very low on memory, with 1454MB remaining.
Sat Jan 09 04:19:35 CET [node_name-01: scsit_lu_0: wafl.memory.statusVeryLowMemory:alert]: WAFL is running very low on memory, with 0MB remaining.
Sat Jan 09 04:57:17 CET [node_name-01: scsit_lu_1: wafl.memory.statusVeryLowMemory:alert]: WAFL is running very low on memory, with 445MB remaining.- A system uptime is higher than 300 days.
Sat Jan 09 01:00:00 CET [node_name-01: statd: kern.uptime.filer:info]: 1:00am up 398 days, 12:12 3 NFS ops, 0 CIFS ops, 0 HTTP ops, 78932960637 FCP ops, 0 iSCSI ops