Cluster communication down after power outage
Applies to
- ONTAP 9
- Switched cluster configurations
- Cluster network switches
- AFF/FAS
Issue
- After a power outage, the nodes in the cluster are not communicating with each other once back online.
- The cluster ports of all the nodes in the cluster are down.
- The cluster applications are not coming online on any of the nodes:
Cluster::> storage failover show
Takeover
Node Partner Possible State Description
-------------- -------------- -------- -------------------------------------
node-01 node-02 true Connected to sanmiguel-02. Waiting
for cluster applications to come
online on the local node. Offline
applications: mgmt, vldb, vifmgr,
bcomd, crs, scsi blade, clam.
node-02 node-01 - Waiting for cluster applications to
come online on the local node
Offline applications: mgmt, vldb,
vifmgr, bcomd, crs, scsi blade, clam.
node-03 - - Node unreachable
node-04 - - Node unreachable
Cluster::> storage failover show-giveback
Partner
Node Aggregate Giveback Status
-------------- ----------------- ---------------------------------------------
node-01
No aggregates to give back
Warning: Unable to list entries on node node-02. RPC: Couldn't make
connection [from mgwd on node "node-01" (VSID: -1) to mgwd at
xxx.xxx.xxx.xxx]
Unable to list entries on node node-03. RPC: Couldn't make
connection [from mgwd on node "node-01" (VSID: -1) to mgwd at
xxx.xxx.xxx.xxx]
Unable to list entries on node node-04. RPC: Couldn't make
connection [from mgwd on node "node-01" (VSID: -1) to mgwd at
xxx.xxx.xxx.xxx]