How to perform graceful shutdown and power up of all ONTAP nodes in a cluster

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 73,287

Visibility:: Public

Votes:: 148

Category:: ontap-9

Specialty:: hw

Last Updated:

Applies to

ONTAP 9
AFF systems
FAS systems
Excludes MetroCluster configurations

Description

Several events might require a graceful shutdown of ONTAP equipment such as:

Scheduled site power outage
Data center-wide maintenance
Physical system move
Preparation for future re-purposing of equipment

Prerequisites

This procedure is for non-MetroCluster configurations only
Local admin credentials for ONTAP
If using NetApp onboard key management (OKM), the cluster-wide passphrase available
Ensure SP/BMC for each controller is accessible
Stop all clients/hosts from accessing data on the NetApp system
Suspend external backup jobs
Personnel onsite to perform physical equipment tasks
General preparation for onsite maintenance
If using NetApp StorageGRID or ONTAP S3 used as FabricPool cloud tier, refer to the Gracefully shutdown and power up your storage system Resolution Guide for the graceful shutdown procedure to perform for those systems after performing the procedure in this article
If using FlexArray array LUNs, follow the specific vendor storage array documentation for the shutdown procedure to perform for those systems after performing the procedure in this article
If using SSDs, refer to SU490: [Impact: Critical] SSD Best Practices: Avoid risk of drive failure and data loss if powered off for more than two months

Prior to shutdown
(Best practices)

Perform additional system health checks
Upgrade ONTAP to a recommended release
Resolve any Active IQ Wellness Alerts and Risks
Run Active IQ Config Advisor
Make a note of any faults presently on the system - amber LEDs on controllers, shelves, IOMs, disks, PSUs, etc.

Identifying hardware components

The ONTAP system is comprised of one or more of the following components. Use the following links for helpful details and images to assist onsite personnel with locating and identifying the equipment.

Controllers:
Disk Shelves
Switches:
- Cluster interconnect
- Cluster management
- Storage
- FC

Procedure

Important: This procedure will shut down all nodes within the cluster and access to data on the cluster will not be possible until the system is powered back up.

Log in to cluster via SSH. Otherwise, log in from any node in the cluster using a local console cable.
Generate a case suppression AutoSupport message for the expected duration of the shutdown event along with any descriptive text:

cluster1::> system node autosupport invoke -node * -type all -message "MAINT=8h Power Maintenance"

Identify the SP/BMC IP address of all nodes:

cluster1::> system service-processor show -node * -fields address node address -------------- ------------ cluster1-01 10.10.10.10 cluster1-02 10.10.10.20 cluster1-03 10.10.10.30 cluster1-04 10.10.10.40

Exit clustershell:

cluster1::> exit

Connect to SP/BMC over SSH using the IP address of any node from Step 3. Otherwise, connect a local console cable to the node. Log in using the same cluster administrator credentials.

If accessing via the SP/BMC prompt, switch to system console and supply the cluster administrator credentials:

login as: admin admin@10.10.10.10's password: <password> SP cluster1-01> system console Type Ctrl-D to exit. SP-login: admin Password: <password> cluster1::>

Note: Open an SSH session window to every SP/BMC for monitoring as described in this step.

Halt all nodes in the cluster:

For most cluster configurations:

cluster1::> system node halt -node * -skip-lif-migration-before-shutdown true -ignore-quorum-warnings true -inhibit-takeover true

For clusters with SnapMirror Synchronous operating in StrictSync mode:

cluster1::> system node halt -node * -skip-lif-migration-before-shutdown true -ignore-quorum-warnings true -inhibit-takeover true -ignore-strict-sync-warnings true

Respond to the prompt for each node:

Warning: Are you sure you want to halt node "cluster1-01"? {y|n}: y Warning: Are you sure you want to halt node "cluster1-02"? {y|n}: y Warning: Are you sure you want to halt node "cluster1-03"? {y|n}: y Warning: Are you sure you want to halt node "cluster1-04"? {y|n}: y 4 entries were acted on.

Wait for each node to halt completely by reaching the LOADER prompt:

LOADER-A>

Connect to each node in the cluster via SP/BMC (if not already connected) or using a local console cable and confirm each node is at the LOADER prompt (as in Step 8).

Physical activity

The physical activity here ensures no equipment damage occurs while the system is powered down and ensures the correct order of equipment startup is followed, so that the ONTAP system is prepared to serve data after the event is complete.

Toggle each PSU rocker switch to the OFF position on each piece of equipment.

Note: Some PSUs do not have rocker switches.

Remove the power cable connecting each PSU to the PDU.
Visually confirm each component has successfully powered off.
Ensure that all controllers, disk shelves, and switches associated with the cluster are powered down.

Power up

When the system is ready to be powered back up following the steps in How to power up all ONTAP nodes in a cluster following a graceful shutdown

Additional Information

Tips

To power off a controller remotely:

SP/BMC> system power off
This will cause a dirty shutdown of your appliance.  Continue? [y/n] y
SP/BMC> system power status
Chassis Power is off

The warning can be ignored only after a clean shutdown and the node is at the LOADER prompt. Any other use can cause loss of data.

Repeat from other SP or BMC in the same chassis (where applicable).

Reference documentation

What is the procedure to gracefully shutdown and power up your storage system during a scheduled power outage?