System status degraded due to ClusterSwitchMissing_Alert
Applies to
- FAS/AFF
- ONTAP 9
- Cluster Network Switches
Issue
- BES-53248 switch is reporting "
ClusterSwitchConnectivity_alert"
Cluster::> alert show
(system health alert show)
Node: Node-01
Alert ID: ClusterSwitchConnectivity_Alert
Resource: Node-01
Severity: Major
Indication Time: Thu Oct 17 21:26:01 2024
Suppress: false
Acknowledge: false
Probable Cause: The ethernet switch, with the name "Switch-SW2",
is not connected to all of the nodes in the cluster.
Possible Effect: If one cluster switch fails, "Node-01" might
lose access to the cluster.
Corrective Actions: Verify that the switch "Switch-SW2" is connected
to the node "Node-01" and that the switch's name has not
been modified due to a change in the switch's link
discovery protocol used for advertisements.
- In EMS we see below events
Node-01: cshmd: hm.alert.raised:alert]: Alert Id = ClusterSwitchConnectivity_Alert , Alerting Resource = Node-01 raised by monitor ethernet-switch
Node-01: mgwd: callhome.hm.alert.major:alert]: Call home for Health Monitor process cshm: ClusterSwitchConnectivity_Alert[Node-02]
- Ping test to the switch from the management LIF is working fine:
Cluster::> ping -lif cluster_mgmt -vserver fas8080-2n-rtp-1 -destination 10.x.x.x
10.x.x.x is alive
- Deleted the reported switch alerts by using the below command, but still the alerts re-appeared after 24hrs
system health alert delete *
- Cluster port e0b of both nodes was down, and these ports were connected to 0/1 and 0/2 of Switch-SW2
Cluster-1::> network port show
Node: Node-01
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status
--------- ------------ ---------------- ---- ---- ----------- --------
e0M Default Default up 1500 auto/1000 healthy
e0a Cluster Cluster up 9000 auto/10000 healthy
e0b Cluster Cluster down 9000 auto/ -
Node: Node-02
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status
--------- ------------ ---------------- ---- ---- ----------- --------
e0M Default Default up 1500 auto/1000 healthy
e0a Cluster Cluster up 9000 auto/10000 healthy
e0b Cluster Cluster down 9000 auto/ -
sw1 >show port-channel brief
Logical Port-Channel Min Link State Trap Type Mbr Ports Active Por
Interface Name Flag
--------- ---------------- --- ---------- -------- ------- ---------- ----------
0/1 ch1 1 Down Disabled Static
0/2 ch1 1 Down Disabled Static
- Tried flapping the ports on the switch, but did not help.
sw1> enable
sw1# configure
sw1# interface 0/1
sw1# shutdown
sw1# exit
sw2> enable
sw2# configure
sw2# interface 0/2
sw2# no shutdown
sw2# exit
- Performed cable reseat but the interface remained down.