Node in switched cluster not serving data after power maintenance
Applies to
- ONTAP
- Cluster interconnect switches
Issue
- After scheduled power maintenance, nodes are powered up but one or more nodes do not serve data.
- Aggregates on non-working nodes report state
unknown
instorage aggregate show
:
Aggregate Size Available Used% State #Vols Nodes RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
root_n1 - - - unknown - cluster1-01 -
aggr_n1 - - - unknown - cluster1-01 -
root_n2 3.04GB 152.4MB 95% online 1 cluster1-02 raid_dp,
normal
aggr_n2 31.77GB 31.75GB 0% online 1 cluster1-02 raid_dp,
- Cluster ports are link down in
network port show
:
Ignore
Speed(Mbps) Health Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status Status
--------- ------------ ---------------- ---- ---- ----------- -------- ------
e0a Cluster Cluster down 9000 auto/- - false
e0b Cluster Cluster down 9000 auto/- - false
- Output of
storage failover show
indicates nodes are waiting for cluster applications to come online:
Takeover
Node Partner Possible State Description
-------------- -------------- -------- -------------------------------------
cluster1-01 cluster1-02 true Connected to cluster1-02
cluster1-02 cluster1-01 true Connected to cluster1-01.
Waiting for cluster applications to
come online on the local node.
Offline applications: vldb, vifmgr,
bcomd, crs, scsi blade, clam.
- First node to come up will typically be cluster master and serving data, whereas other nodes are powered up but cluster apps are offline and not serving data.