Multiple panics in process NwkThd_05 due to Fpolicy memory leak
Applies to
- ONTAP 9
- System Panic
- Fpolicy
Issue
- System panicked multiple times due to hung network threads
- The following can be seen multiple times prior to the panic in the
EMS
log via autosupport orevent log show:
cluster-n01: sched_monitor: mgr.stack.longrun.proc:NOTICE]: Long running process: NwkThd_05
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 9: nbladekmod.ko::_ZN8HashListI14NfsFPolicyOpId12NfsFPolicyOpLb1EN4util10RwSpinLockELj256ELj16EE4findERKS0_RPNS4_13HashListEntryE(0x0xffffffffa10cce00lx) + 0x0xce
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 12: nbladekmod.ko::Nfs3ProcWrite(0x0xffffffffa10c2840lx) + 0x0x390
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 13: nbladekmod.ko::_Z18OncRpcAuthenticateP20onc_rpc_exec_contextPFvS0_E(0x0xffffffffa10c7c50lx) + 0x0x47a
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 14: nbladekmod.ko::_Z24OncRpcParseRequestHeaderP20onc_rpc_exec_contextP13RewindContextP18onc_rpc_connectionP17InputBufferList_t(0x0xffffffffa10c9600lx) + 0x0xa81
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 15: nbladekmod.ko::_Z10TcpProcessP17stream_connection(0x0xffffffffa10b4440lx) + 0x0x153
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 16: nbladekmod.ko::_Z14NfsReceiveDataP3MQE(0x0xffffffffa107be90lx) + 0x0x6f
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 17: nbladekmod.ko::NfsEntry(0x0xffffffffa107bfb0lx) + 0x0xaf
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 18: nbladekmod.ko::NfsNotify(0x0xffffffffa107c260lx) + 0x0x1e8
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 19: nbladekmod.ko::_ZN3PCP12SoRecvUpcallEPv(0x0xffffffffa10b67c0lx) + 0x0x6c
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 20: nbladekmod.ko::_ZN3PCP21SoSendUpcallFcEnabledEPv(0x0xffffffffa10b69e0lx) + 0x0xc6
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 21: kernel::BSDNET_net_is_ip_alive(0x0xffffffff80b9f520lx) + 0x0x99e
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 22: kernel::vsun_register(0x0xffffffff80bd5750lx) + 0x0x3b8
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 23: kernel::sowakeup(0x0xffffffff807365c0lx) + 0x0x6e
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 24: kernel::tcp_do_segment(0x0xffffffff80805620lx) + 0x0x1550
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 25: kernel::tcp_input(0x0xffffffff80803110lx) + 0x0x1b36
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 26: kernel::ip_input(0x0xffffffff807f55e0lx) + 0x0x3c5
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 27: kernel::cg_get_cg_space_for_ports(0x0xffffffff80bd8350lx) + 0x0xd97
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 28: kernel::netapp_dispatch_mb_cache_objs(0x0xffffffff80bd9490lx) + 0x0x24b
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 29: kernel::ql_rcv_isr(0x0xffffffff80a7d390lx) + 0x0xe61
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 30: kernel::ql_isr(0x0xffffffff80a7f4a0lx) + 0x0x11c
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 31: kernel::cg_print(0x0xffffffff80ba5800lx) + 0x0x71e
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 32: kernel::cg_nwk_thread_pool_create(0x0xffffffff80badcd0lx) + 0x0xd8b
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 33: kernel::fork_exit(0x0xffffffff80642eb0lx) + 0x0xc7
[cluster-n01: sched_monitor: mgr.stack.frame:NOTICE]: Stack frame 34: kernel::fork_trampoline(0x0xffffffff809febc0lx) + 0x0xe