Skip to main content
NetApp Knowledge Base

What is the LIF Choice order for Cluster Aware backups in NDMP?

  1. Last updated
  2. Save as PDF
Views:
6,376
Visibility:
Public
Votes:
12
Category:
ontap-9
Specialty:
dp
Last Updated:

​ ​ ​  

Applies to

  • ONTAP 9
  • NDMP
  • Data Management Application (DMA) = Backup Application
  • Cluster Aware Backup (CAB)

Answer

There are two types of communication that are used by NDMP Backups configured with CAB: Control connection and Data connection.

Control connection:

  • The initial connection made by the DMA (Backup Application), used to negotiate backup settings, parameters, CAB configuration.
  • Within NDMP logs this is the following communication
    • DMA<<S or DMA>>S
    • S < BKUP
  • This communication always uses the interface which the DMA selects.
    • I.E. The DMA targets an NDMP host for backup using a specific IP address.
  • As long as the port is configured to listen for NDMP requests on port 10000, then this can be established.​​User-added image
    • Whether the port is able to listen for this communication depends on the SVM which owns this LIF and if the same SVM has NDMP enabled:

Data Connection:

  • This is the connection established for the purpose of sending the Backup Data (Data that is Dumped) to a remote address specified by the DMA during the initial Control Communication.
  • NDMP debug log example:

Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG: DMA>>S V4 sequence=10 (0xa)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      Time_stamp=0x585a5525 (Dec 21 10:10:45 2016)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      message type=0 (NDMP4_MESSAGE_REQUEST)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      message_code=0x40a (NDMP4_DATA_CONNECT)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      reply_sequence=0 (0x0)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      error_code=0 (NDMP4_NO_ERR)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      addr_type=1 (NDMP4_ADDR_TCP)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      tcp_addr len=1 (0x1)
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      tcp_addr[0] ip_addr=0xc0a8002a [192.168.0.42]
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      tcp_addr[0] port=4477
Wed Dec 21 2016 10:10:32 +00:00 [kern_ndmpd:info:5123] [46049]  DEBUG:      tcp_addr[0] addr_env_len=0 (0x0)

Note: The LIF used by the DMA to set up the initial control communication is not always the LIF used for the Data Connection.

What Data LIF will be used for this DATA_CONNECT operation?

  • The Interface used for the Data Connection cannot be explicitly set but can be isolated to a specific set of interfaces that are possible.
  • This creates some challenges and should be understood to best isolate & guarantee the desired interface is used:

Based on the SVM you initially communicate with for the Control communication, you will use this SVM's interfaces based on the corresponding preferred-interface-role:

cm2520-ams::> vserver services ndmp show -vserver cm2520-ams -fields preferred-interface-role

vserver    preferred-interface-role

---------- -----------------------------------

cm2520-ams intercluster,cluster-mgmt,node-mgmt

  • The order defined here is the order in which interfaces owned by the corresponding SVM will be used.
  1. Intercluster
  2. If intercluster fails or does not exist, use cluster-mgmt,
  3. if cluster-mgmt fails or does not exist, use node-mgmt
  • When control communication goes initially through a Data SVM, there are only two possible interfaces for the Data Connection: Data, Intercluster

cm2520-ams::> vserver services ndmp show -vserver iscsi -fields preferred-interface-role

vserver preferred-interface-role

------- ------------------------

iscsi   intercluster,data

  • Because CAB checks for volume affinity (which node the volume resides in), CAB will always use interfaces that are also on the same Node.
    • If the preferred interface does not exist within the node, the next preferred interface is chosen.
  • Special Cases: Multiple Interfaces available to choose from with Preferred interface role choice.
    • Based on the Preferred Interface Role chosen, there may often be more than one interface available of that particular role, on that particular node.

Example: With data Interfaces:

User-added image

User-added image

  • Problem: It might be desirable to use  'iscsi_ndmp_admin' on port e0e for NDMP communication.
    • The order in which the interface is selected cannot be configured, but we do know which interface will be used first based on the current port used by the LIF &  the output of ifconfig -a. 
    • The order in which the interface is used depends on the order in which the interface appears in the node-run output of ifconfig -a.

cm2520-ams::> node run -node cm2520-ams-02 ifconfig -a

User-added image

  • Because e0c appears before e0e, the LIF's iscsi_lif_n2 & iscsi_nfs will be used first for DATA_CONNECT.
  • DATA_CONNECT MUST FAIL on these first two LIF's for the 3rd LIF  iscsi_ndmp_admin to be used.
  • As all these LIFs reside within the same SUBNET, the likelihood is that the first or second would be successful.
  • Sharing the subnet & interface could intrude on existing production data connections.
  • Note: It is best practice to separate Backup traffic from Production traffic within different subnets, to prevent congestion, interference, and in this case wrong LIF choice.
  • How can this be solved?
    • Configuring the existing LIF iscsi_ndmp_admin and BKUP destination within an isolated subnet, which cannot be reached by the first two interfaces.
      • This would ensure the desired interface is used.
    • Moving the existing LIF iscsi_ndmp_admin to port e0a would ensure this LIF is chosen first during the DATA_CONNECT.
    • It may be better to utilized Intercluster LIFs to reduce the number of LIF's which may be attempted during the Backup.
      • If existing InterCluster LIFs cannot reach the Backup destination because of network limitations, configuring a new InterCluster LIFs within new IPSpaces may be required.
      • When new InterCluster LIFs are configured in new IPSpaces, existing Cluster Peering will not be affected.
    • In this case, if e0a is used by a new or existing InterCluster LIF within the Default IPSpace, it would be picked first if the SVM was configured to look for InterCluster LIFs first:

User-added image

Additional Information

For more information on how NDMP works with LIFs in non-default IP spaces, see "NDMP Cluster Aware Backup cannot use Intercluster LIF's both when LIF or SVM reside within non-default IPSpace"

Note: The VLANs order in the ifconfig -a is based on the creation order

Note: If intercluster is the preferred interface role, ensure that the intercluster LIF type is hosted on all nodes of the cluster that hosts the volume being backed up

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.