All Paths Down due to LIFs manually offlined
Applies to
- ONTAP 9
- VMware ESXi
Issue
- All Paths Down reported by VMware in ESXi's
vobd.log
:
2024-10-31T14:15:27.589Z In(14) vobd[2097549]: [APDCorrelator] 2687006451996us: [vob.storage.apd.start] Device or filesystem with identifier [naa.600a09803831482f393f543673376266] has entered the All Paths Down state.
vmkernel.log
shows ports going down before:
2024-10-31T14:15:18.214Z In(182) vmkernel: cpu27:2121420)qlnativefc: vmhba64(13:0.0): qlnativefcAsyncEvent:1064:RSCN database changed -- 0002 1301 0000.
2024-10-31T14:15:18.228Z In(182) vmkernel: cpu14:2097842)qlnativefc: vmhba64(13:0.0): qlnativefcUnregSess:6607:qlnativefcUnregSess sess 0x4312535d28f0 for deletion 20:02:d0:39:ea:01:23:45
2024-10-31T14:15:18.228Z In(182) vmkernel: cpu8:2097841)qlnativefc: vmhba64(13:0.0): qlnativefcLogoutIocb:3073:qlnativefcLogoutIocb: Freeing nport handle
2024-10-31T14:15:18.228Z In(182) vmkernel: cpu7:2121415)qlnativefc: vmhba64(13:0.0): qlnativefcStatusEntry:2036:(1:0) Port down status: port-state=0x3
- Looking at ONTAP
audit.log
, a command was issued downing multiple SVMs
00000014.0077bbb4 0b8a3ce9 Thu Oct 31 2024 14:15:18 +00:00 [kern_audit:info:2263] 8003e80000296e4e:8003e80000296eee :: cluster1:ssh :: 10.0.0.1:50814 :: cluster1:admin :: network interface modify -vserver *svm* -status-admin down -lif * :: Success: 15 entries were modified.