FAS8300, FAS8700, AFF/ASA C400 onboard ports go down, leading to takeover
Applies to
- FAS8300
- FAS8700
- AFF A400, AFF C400
- ASA A400, ASA C400
Issue
- Link down errors are seen on the onboard e0a/e0b/e0c/e0d Ethernet ports on the FAS8300, FAS8700, AFF A400, AFF C400, ASA A400 or AFF C400 system, followed by a node takeover.
- 25GbE Ethernet ports e0a and e0b are used for HA interconnect
- 40/100GbE Ethernet ports e0c and e0d are typically used for cluster interconnect (except for AFF A400 and ASA A400 with X1151A in slot 3)
- The following errors are seen:
Fri Oct 08 00:37:27 -0400 [node-01: kernel: netif.linkDown:info]: Ethernet e0a: Link down, check cable.
Fri Oct 08 00:37:27 -0400 [node-01: intr: rlib.ifconfig.linkEvent:notice]: params: {'eventType': 'DOWN', 'ifname': 'e0a'}
Fri Oct 08 00:37:27 -0400 [node-01: kernel: netif.linkDown:info]: Ethernet e0b: Link down, check cable.
Fri Oct 08 00:37:27 -0400 [node-01: intr: rlib.ifconfig.linkEvent:notice]: params: {'eventType': 'DOWN', 'ifname': 'e0b'
Fri Oct 08 00:37:27 -0400 [node-01: kernel: netif.linkDown:info]: Ethernet e0c: Link down, check cable.
Fri Oct 08 00:37:27 -0400 [node-01: kernel: netif.linkDown:info]: Ethernet e0d: Link down, check cable.
Fri Oct 08 00:37:28 -0400 [node-01: cf_main: cf.fsm.takeoverByPartnerDisabled:error]: Failover monitor: takeover of node-01 by node-02 disabled (HA interconnect error. Verify that the partner node is running and that the HA interconnect cabling is correct, if applicable. For further assistance, contact technical support).
Fri Oct 08 00:37:28 -0400 [node-01: cf_firmware: cf.fm.partnerFwTransition:info]: params: {'progresscounter': '0', 'newstate': 'SF_UNKNOWN', 'prevstate': 'SF_UP'}
Fri Oct 08 00:37:30 -0400 [node-01: nvmm_mirror_sync: nvmm.mirror.aborting:debug]: mirror of sysid 1, partner_type HA Partner and mirror state NVMM_MIRROR_LAYOUT_SYNCING is aborted because of reason NVPM_ERR_MSG_SEND_FAILED.
Fri Oct 08 00:37:30 -0400 [node-01: vifmgr: vifmgr.portdown:notice]: A link down event was received on node node-01, port e0c.
Fri Oct 08 00:37:30 -0400 [node-01: vifmgr: vifmgr.clus.linkdown:EMERGENCY]: The cluster port e0c on node node-01 has gone down unexpectedly.
Fri Oct 08 00:37:32 -0400 [node-01: cf_main: cf.fsm.partnerNotResponding:notice]: Failover monitor: partner not responding
Fri Oct 08 00:37:32 -0400 [node-01: cf_main: cf.fsm.takeoverCountdown:info]: Failover monitor: takeover scheduled in 10 seconds
Fri Oct 08 00:37:42 -0400 [node-01: cf_main: cf.fsm.takeover.noHeartbeat:alert]: Failover monitor: Takeover initiated after no heartbeat was detected from the partner node.
Fri Oct 08 00:37:42 -0400 [node-01: cf_main: cf.fsm.stateTransit:info]: Failover monitor: UP --> TAKEOVER
