Vifmgr got OOQ with duplicate IP error
Applies to
Issue
- The LIF was migrated due to a problem with vifmgr
- Vifmgr got OOQ with duplicate IP error
EMS:
[?] Sat Oct 22 20:18:18 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xx.xxx.x!! sent from Ethernet address: 02:a0:98:a3:44:4d.
[?] Sat Oct 22 20:18:18 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xxx.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:51.
[?] Sat Oct 22 20:18:18 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xxx.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:51.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xx.x.xx!! sent from Ethernet address: 02:a0:98:a3:44:52.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xxx.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:51.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xx.xx.x!! sent from Ethernet address: 02:a0:98:a3:44:4d.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.x.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:4d.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xxx.xxx.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:4d.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xxx.xxx.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:4d.
[?] Sat Oct 22 20:18:19 +0900 [node2: kernel: netinet.ethr.duplct.ipAdrr:error]: Duplicate IP address xx.xxx.xx.xxx!! sent from Ethernet address: 02:a0:98:a3:44:51.
[?] Sat Oct 22 20:18:37 +0900 [node2: wafl_exempt03: ems.engine.suppressed:debug]: Event 'wafl.vol.runningOutOfInodes' suppressed 1039 times in last 1822 seconds.
[?] Sat Oct 22 20:18:37 +0900 [node2: wafl_exempt03: wafl.vol.runningOutOfInodes:error]: The file system on Volume sbc_sdsvgdc_d@vserver:ee6b63bc-04c7-11e9-b2c7-00a098a3443f is using 80 percent or more of the files that can be contained on the volume.
[?] Sat Oct 22 20:18:40 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF OC1_N2_PIH_data (on virtual server 24), IP address xx.xx.x.xx, is being removed from node node2, port a0a-47.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF OC1_N2_SDS_data (on virtual server 4), IP address xx.xxx.xx.xxx, is being removed from node node2, port a1a-711.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF ON3_SDS01_data (on virtual server 15), IP address xx.xxx.xx.xxx, is being removed from node node2, port a1a-711.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF ON4_SDS01_data (on virtual server 17), IP address xx.xx.xx.xxx, is being removed from node node2, port a1a-711.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF OC1_N2_PORTAL_data (on virtual server 31), IP address xx.xxx.xx.xxx, is being removed from node node2, port a1a-711.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF OC1_N2_ETCn_data (on virtual server 6), IP address xxx.xxx.xx.xxx, is being removed from node node2, port a2a-132.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF ON4_ETC01_data (on virtual server 16), IP address xxx.xxx.xx.xxx, is being removed from node node2, port a2a-132.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF ON4_SWS01_data (on virtual server 18), IP address xx.xx.xx.x, is being removed from node node2, port a2a-2002.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF ON4_SCG01_data (on virtual server 20), IP address xx.xx.xxx.x, is being removed from node node2, port a2a-2004.
[?] Sat Oct 22 20:18:41 +0900 [node2: vifmgr: vifmgr.lifBeingRemoved:notice]: LIF ON4_CARD01_data (on virtual server 19), IP address xx.x.xx.xxx, is being removed from node node2, port a2a-21.
vifmgr:
00000015.003969f2 22f47aea Sat Oct 22 2022 20:17:04 +09:00 [kern_vifmgr:info:6697] ******* Secondary OOQ mtrace dump BEGIN *********
00000015.00396bbc 22f47b89 Sat Oct 22 2022 20:17:18 +09:00 [kern_vifmgr:info:6697] Notice: online_status_callback: RDB unit is offline
00000015.00399db8 22f47e39 Sat Oct 22 2022 20:18:27 +09:00 [kern_vifmgr:info:6697] [0x809b9be00] [EventMgr::unitOffline] Setting VifMgr operational status as OOQ
00000015.00399db9 22f47e39 Sat Oct 22 2022 20:18:27 +09:00 [kern_vifmgr:info:6697] [0x809b9be00] [FailoverMgr::localNodeDown] VifMgr on node node2 is now out of quorum.
00000015.0039a418 22f47e8f Sat Oct 22 2022 20:18:39 +09:00 [kern_vifmgr:info:6697] [0x80ac6a200] [DelayNotice::~DelayNotice] long-running operation (L2PingHealthMonitor::LocalReachabilityData::initBroadcastDomainData): 118118 milliseconds elapsed (> 5000)
00000015.0039a9e7 22f47ec3 Sat Oct 22 2022 20:18:41 +09:00 [kern_vifmgr:info:6697] [0x80af33800] [VserverCache::get_vserver_name] Failed to look up name for vserver id 4294967293: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 2 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
00000015.0039a9e9 22f47ec3 Sat Oct 22 2022 20:18:41 +09:00 [kern_vifmgr:info:6697] [0x80af2de00] [VserverCache::get_vserver_name] Failed to look up name for vserver id 4294967293: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 2 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
00000015.0039a9eb 22f47ec3 Sat Oct 22 2022 20:18:42 +09:00 [kern_vifmgr:info:6697] [0x809b9be00] [EventMgr::workerThreadMain] long-running event: event="Unit is offline", time=14926 ms
00000015.0039aa5b 22f47ec3 Sat Oct 22 2022 20:18:43 +09:00 [kern_vifmgr:info:6697] [0x80af2de00] [VserverCache::get_vserver_name] Failed to look up name for vserver id 4294967293: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 2 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
00000015.0039b07e 22f47ef9 Sat Oct 22 2022 20:18:51 +09:00 [kern_vifmgr:info:6697] [0x80af36500] [vifmgr_virtual_interface_iterator::get_imp] Failed to successfully calculate operational status: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 25 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
00000015.0039b07f 22f47ef9 Sat Oct 22 2022 20:18:51 +09:00 [kern_vifmgr:info:6697] [0x80adba000] [vifmgr_virtual_interface_iterator::get_imp] Failed to successfully calculate operational status: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 25 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
00000015.0039b081 22f47ef9 Sat Oct 22 2022 20:18:51 +09:00 [kern_vifmgr:info:6697] [0x80ac67500] [vifmgr_virtual_interface_iterator::get_imp] Failed to successfully calculate operational status: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 25 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
00000015.0039b083 22f47ef9 Sat Oct 22 2022 20:18:51 +09:00 [kern_vifmgr:info:6697] [0x80ac75600] [vifmgr_virtual_interface_iterator::get_imp] Failed to successfully calculate operational status: Timeout: Operation "vserver_info_iterator::get_imp()" took longer than 25 seconds to complete [from vifmgr on node "node2" (VSID: -1) to mgwd at 127.0.0.1]
- The root aggregate has only one disk, which is busy when a problem occurs