Skip to main content
NetApp Knowledge Base

NDMP dump and restore phases description

Views:
4,864
Visibility:
Public
Votes:
2
Category:
ndmp
Specialty:
dp
Last Updated:

 

Applies to

  • Data ONTAP 7mode
  • clustered Data ONTAP 8
  • ONTAP 9

Answer

Terms

Some key terms used in the description of the dump phases are:

  • Inode:
    • Every file on a file system has an identifier associated with it.
      • This id is called the inode.
    • A file system typically has inodes pre-allocated on it.
    • The output for df -i in ONTAP shows volume's total number of used inodes and total number of free inodes.
    • For the purpose of this document we will use the terms inode and file interchangeably.
  • Inode file:
    • This is a special file on the file system that contains a list of all the inodes in a volume and their details.
  • Inode map:
    • An array that has as many elements as the number of inodes on the volume.
    • The inode number serves as the index for its corresponding entry in the array.
    • A value of 1 in any entry indicates that the corresponding file will be present in a given backup.
  • Offset map:
    • An array that has as many elements as the number of inodes on the volume.
    • The inode number serves as the index for its corresponding entry in the array.
    • If an inode exists in a backup, its corresponding entry contains the physical address on tape that marks the beginning of the file data within the backup image.
Dump phases
  • For backups that are initiated by NDMP with file history turned on, dump must generate File History in Phase III and IV as well as the offset Map in Phase IIIb.
  • Generating file history and the offset map adds some cost to NDMP backups in these Phases, but allows NDMP to provide invaluable features such as backup indexing, DAR, and Enhanced DAR.
  • For more information about file history, see FAQ: NDMP File History.

Phase I

  • dump generates the list of files that need to be backed up.
  • The output of this phase is what is called an inode map.
  • The inode map contains an entry for every inode on the volume.
  • The inode that is to be backed up has its corresponding entry set to 1 whereas the inode that is not to be backed up has its entry set to 0.

Note: The more inodes there are, the longer Phase I will take to complete.  For example, a volume with 1 billion inodes would take significantly longer than a volume with 10 million inodes.

Phase II

  • dump writes the inode map generated in Phase I to the tape.

Phase III

  • dump writes the entire directory structure for the backup dataset to tape.
  • If enabled, file history for directories is generated and communicated to the backup application during this phase.
    • Phase IIIa:
      • ACLs Phase: In this phase dump backs up ACLs for the data set to the tape.
      • This step may take more time if a lot of files within the data set have ACLs.
    • Phase IIIb:
      • This Phase is executed only for NDMP backups that have File History turned on.
      • The output of this phase is the offset map.
      • For each file on any given backup, the offset map contains the physical address on the tape that marks the beginning of the file within the backup image.

Note:  The more directory inodes there are, the longer Phase III will take to complete.  In addition, because dump traverses the entire directory tree, a deeper, wider, or both directory structure can take longer to read than a simpler directory structure.

Phase IV

  • This phase dumps the actual file data on tape.
  • This phase operates in the inode order.
  • As a result, a smaller inode number is guaranteed to be found before a larger inode number.
  • If enabled, file history for files is generated and communicated to the backup application during this phase.

Phase V

  • This is a duplicate of Phase IIIa.
  • This is what traditionally existed in NetApp native dump.
  • This phase is retained for reasons of backward compatibility, but is a no-op in modern versions of ONTAP.
Restore phases
  • Phase I:
    • Entire backup image's directory structure is read into an inofile, and file info is read into the inomap.
  • Phase II:
    • All files and directories selected for the restore are created on the filesystem, based on the inofile and inomap created on Phase I. 
    • Due to the nature of restore operations, files and directories created during Phase II will be displayed as zero byte when the file system is browsed or listed
    • The zero byte file and directory entries will change to non-zero values, and sizes may be observed to increase after the restore enters Phase III
  • Phase III:
    • The contents of individual files are read from the backup image, and the tree structure created in Phase II is populated with data. 
Qtree/Volume dump vs. non-Qtree sub-directory dump
  • Phase I of dump behaves differently based on whether the user initiated the backup of a Volume/Qtree or a non-Qtree sub-directory/subtree.
  • With sub-directory dumps, each directory and file below the specified root of the backup must be examined to determine whether it should be included in the backup.
  • This is a time-consuming operation especially if the sub-directory contains several million inodes (files).
  • With Volume and Qtree based backups, it is not necessary to go through the entire directory tree to determine what needs to be backed up.
  • Instead ONTAP can go through the inode file for the backup snapshot.
  • By just looking at the inode file, it is possible to determine whether the inode exists and whether it is a part of a given qtree/volume.
  • Using the inode file is a much faster and efficient process as compared to the directory traversal.
  • The inode file does not contain enough information to be able to do the same for non-qtree sub-directory dumps, so it is recommended to backup Qtrees and Volumes whenever possible rather than non-Qtree sub-directories.
  • Note that using the exclude NDMP environment variable causes Qtree and Volume dumps to perform the slower directory traversal method as well.
  • If backup speed is a factor, avoid using exclusions in NDMP-initiated dump backups..

Additional Information

FAQ: NDMP File history

 

 

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.