CLAM panic after adding A250 nodes to existing cluster through BES-53248 switch
Applies to
- Broadcom BES-53248 switch purchased from NetApp
- Cluster expansion adding nodes that use shared cluster/HA ports:
- AFF A320
- AFF A250
- FAS500f
Issue
- While adding AFF A250 nodes to existing cluster through BES-53248 switch, cabling new nodes to cluster switch causes cluster ports to go down and CLAM panic:
May 08 00:10:01 [cluster-01:vifmgr.clus.linkdown:EMERGENCY]: The cluster port e0a on node cluster-01 has gone down unexpectedly.
May 08 00:11:13 [cluster-01:vifmgr.clus.linkdown:EMERGENCY]: The cluster port e0b on node cluster-01 has gone down unexpectedly.
PANIC : Received PANIC packet from partner, receiving message is (Coredump and takeover initiated because Connectivity, Liveliness and Availability Monitor (CLAM) has determined this node is out of quorum.)
- Existing node cluster ports connected to ports 0/1 and 0/2 on BES-53248 switches with 10G speed
- AFF A250 nodes are plugged into switch ports 0/3 and 0/4 at 25G speed
- Existing nodes will show cluster ports offline and waiting for cluster applications to come online:
Takeover
Node Partner Possible State Description
-------------- -------------- -------- -------------------------------------
cluster1-01 cluster1-02 - Waiting for cluster applications to
come online on the local node
Offline applications: mgmt, vldb,
vifmgr, bcomd, crs, scsi blade, clam.
cluster1-02 cluster1-01 true Connected to cluster1-01, Partial
giveback
cluster1::> net int show
(network interface show)
Logical Status Network Current Current Is
Vserver Interface Admin/Oper Address/Mask Node Port Home
----------- ---------- ---------- ------------------ ------------- ------- ----
Cluster
cluster1-01_clus1
up/- 169.254.11.190/16 cluster1-01 e0c false
cluster1-01_clus2
up/- 169.254.122.210/16 cluster1-01 e0c true
cluster1-02_clus1
up/down 169.254.10.90/16 cluster1-02 e0c false
cluster1-02_clus2
up/down 169.254.40.40/16 cluster1-02 e0c true
- Switch logs show the existing cluster ports go down as soon as new nodes are connected:
<189> May 7 16:05:42 switch1 TRAPMGR[trapTask]: traputil.c(721) 15624 %% NOTE Link Down: 0/1
<189> May 7 16:05:42 switch1 TRAPMGR[trapTask]: traputil.c(721) 15623 %% NOTE Link Down: 0/2
<189> May 7 16:05:42 switch1 TRAPMGR[trapTask]: traputil.c(721) 15620 %% NOTE SFP inserted in 0/3