Network ports down in ifgroup after node or switch reboot
Applies to
- FAS8200
- Cisco N5K-C5548UP running NX-OS 7.3(7)N1(1b))
- Ifgroups with 10GbE optical ports for CNA EP 8324 and 40GbE breakout optical cables.
- Ontap 9
Issue
- Multiple ports down when rebooting the node or the switch:
...
Sun Mar 21 10:40:42 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0m, port e0d has transitioned to the active state.
Sun Mar 21 10:40:42 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0m, port e0c has transitioned to the active state.
Sun Mar 21 10:40:46 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0d, port e1c has transitioned to the active state.
Sun Mar 21 10:44:16 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0c, port e0e has transitioned to the active state.
Sun Mar 21 10:44:18 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0r, port e0h has transitioned to the active state.
Sun Mar 21 10:44:18 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a3p, port e0f has transitioned to the active state.
Sun Mar 21 10:44:19 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a3p, port e1b has transitioned to the active state.
Sun Mar 21 10:46:23 +0100 [node_name: intr: net.ifgrp.lacp.link.inactive:error]: ifgrp a0m, port e0c has transitioned to an inactive state. The interface group is in a degraded state.
Sun Mar 21 10:46:23 +0100 [node_name: intr: net.ifgrp.lacp.link.inactive:error]: ifgrp a0m, port e0d has transitioned to an inactive state. The interface group is in a degraded state.
Sun Mar 21 10:46:23 +0100 [node_name: intr: net.ifgrp.lacp.link.inactive:error]: ifgrp a0c, port e0e has transitioned to an inactive state. The interface group is in a degraded state.
Sun Mar 21 10:46:24 +0100 [node_name: intr: net.ifgrp.lacp.link.inactive:error]: ifgrp a3p, port e0f has transitioned to an inactive state. The interface group is in a degraded state.
Sun Mar 21 10:46:24 +0100 [node_name: intr: net.ifgrp.lacp.link.inactive:error]: ifgrp a0d, port e0g has transitioned to an inactive state. The interface group is in a degraded state.
Sun Mar 21 10:46:24 +0100 [node_name: intr: net.ifgrp.lacp.link.inactive:error]: ifgrp a0r, port e0h has transitioned to an inactive state. The interface group is in a degraded state.
Sun Mar 21 10:46:33 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0m, port e0c has transitioned to the active state.
Sun Mar 21 10:46:34 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0m, port e0d has transitioned to the active state.
Sun Mar 21 10:46:42 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0d, port e0g has transitioned to the active state.
Sun Mar 21 10:46:50 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a0c, port e0e has transitioned to the active state.
Sun Mar 21 10:46:52 +0100 [node_name: intr: net.ifgrp.lacp.link.active:notice]: ifgrp a3p, port e0f has transitioned to the active state.
...
Ifconfig -vvv
output example:
a0c: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 02:a0:98:fb:89:71
hwaddr 02:a0:98:fb:89:71
pcp 1
laggproto lacp lagghash l3
lagg options:
flags=1<USE_FLOWID>
flowid_shift: 16
lagg statistics:
active ports: 1
flapping: 0
lag id: [(8000,02-A0-98-FB-89-71,0032,0000,0000),
(4E20,00-23-04-EE-BF-2F,040C,0000,0000)]
laggport: e1a flags=20000<DEFAULT> state=41<ACTIVITY,DEFAULTED>
[(8000,02-A0-98-FB-89-71,8008,8000,0008),
(FFFF,00-00-00-00-00-00,0000,FFFF,0000)]
input/output LACPDUs: 0 / 0
laggport: e0e flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
[(8000,02-A0-98-FB-89-71,0032,8000,000D),
(4E20,00-23-04-EE-BF-2F,040C,8000,261F)]
input/output LACPDUs: 45 / 64
groups: lagg
media: Ethernet autoselect
status: active
- Ports recovered when removed and re-added to the ifgroup.