How to configure SNMP monitoring on DATA ONTAP

Applies to

• Clustered Data ONTAP 8
• ONTAP 9

Answer

Note: See the following for managing SNMP in newer versions of ONTAP: Command for managing SNMP

• A brief introduction of Simple Network Management Protocol (SNMP) and SNMP traps in clustered Data ONTAP.
• How to use SNMP to get information from C-Mode cluster systems?
• How to configure SNMP traps and receive events on the desired clients?

Note: Refer to the following TR-Guide - SNMP Support in Data ONTAP

The purpose of this report is to help customers and NetApp field teams to understand the level of SNMP support in Clustered Data ONTAP 8.2.x, 8.3.x, and ONTAP 9.  It also compares the OID availability for both 7-mode and Cluster mode. It provides information at the individual table or group level only. The netapp.mib file provides additional information about the various fields or variables supported within a table or group. A MIB browser tool like iReasoning can be used to easily interpret the contents of the netapp.mib file. 

SNMP:

SNMP is a widely used network monitoring and control protocol. Data is passed from SNMP agents, which are hardware and/or software processes reporting activity in each network device (hub, router, or bridge) to the workstation console used to oversee the network. The agents return information contained in a Management Information Base (MIB), which is a data structure that defines what is obtainable from the device and what can be controlled (turned off or on). Originating in the UNIX community, SNMP has become widely used on all major platforms.
MIBs describe the structure of the management data of a device subsystem; they use a hierarchical namespace containing object identifiers (OID). Each OID identifies a variable that can be read or set via SNMP.

Note: NetApp does not support snmp-set operations. Also, SNMP-support is only cluster-wide and is not vserverized. However, this will be done in releases after 8.1 and is different from 7G vfilers, as snmp-support was never vfilerized.

Enabling/Disabling SNMPs:

The SNMP protocol can be enabled and disabled on a cluster using CLIs/ZAPIs:

• Enabling SNMP on a cluster using CLI - From ngsh, run options-option-name snmp.enable-option-value on
• Disabling SNMP on a cluster using CLI - From ngsh, run options-option-name snmp.enable-option-value off
• Enabling SNMP on a cluster using ZAPI - Using ontapi or zexplore.exe, run the API snmp-enable
• Disabling SNMP on a cluster using ZAPI - Using ontapi or zexplore.exe, run the API snmp-disable
  
  Example: $> ontapi snmp-[en|dis]able
  Note: SNMP protocol can be enabled or disabled only cluster-wide. SNMP to an individual node in a cluster is not possible in Data ONTAP 8.1 C-Mode .

7G-like SNMP UI:

The Data ONTAP C-Mode leverages a bunch of UIs (CLIs and ZAPIs) to configure the SNMP details on the cluster. Given below is a brief on each CLI that helps configure SNMP on a clustered system:

• snmp contact: View or modify the contact details
• snmp location: View or modify the location details
• snmp init: Enables or disables traps sent out from the cluster [1-> enabled, 0->disabled]
• snmp authtrap: Enables or disables authenticationFailure traps [1->enabled,0->disabled]
• snmp community add|delete: To view, add or delete the communities in the cluster, there will be a default 'public' community which is ro.
• Note: Only readonly communities are supported. Also, in order to delete the first snmp community entry, the user needs to delete the traphost that is used for notification purposes.
• snmp traphost add|delete: To view, add or delete the traphosts in the cluster, all the traps (or events) that occur in the cluster are sent to these hosts when the hosts are running snmptrapd
• options snmp.enable: Enables or disables snmp protocol on the cluster [on=>enabled, off=>disabled]

The following is a brief on each ZAPI that helps configure SNMP on a clustered Data ONTAP system:

• snmp-enable: Enables the snmp protocol on the cluster
• snmp-disable: Disables the snmp protocol on the cluster
• snmp-trap-enable: Enables traps being sent out to the traphosts
• snmp-trap-disable: Disables traps being sent out to the traphosts
• snmp-community-add: Adds the snmp community. Only ro communities are supported in Data ONTAP 8.1 C-Mode
• snmp-community-delete: Deletes an existing community
• snmp-traphost-add: Adds a traphost
• snmp-traphost-delete: Deletes an existing traphost
• snmp-status: Gives the details of the snmp configuration on the cluster (Such as location, contact, traps, traphosts, communities details)
• snmp-get: Reads the object value when a OID is given as input to the API (Similar to snmpget unix utility)
• snmp-get-next: Reads the value of the object next to what is given as OID (Similar to snmpgetnext unix utility)
• Add/modify/view contact: The contact and location details of the cluster can be added or modified using snmp CLI. There is no corresponding API for modifying the contact or location details. However, the cluster details can be read (cannot be modified using snmp or ZAPI) using snmp[walk|get|getnext] (or) snmp-status

Example:

Add/Modify using CLI :

Read using ZAPI call :
$> ontapi -x snmp-status

Read using snmp calls :
$> snmpwalk -c public -v [1|2c]

How to configure snmpv3 on C-Mode systems:

• Create an snmpv3 user on the cluster using the security login create CLI.
• Enter the EngineID (use the local EngineID, which is taken as default).
• Enter the auth protocol and enter the password for the specified snmpv3 user.
• Run the snmp[walk|get|getnext] to the snmpv3 user, by specifying -v 3 and providing user credentials.

The appendix section in the PDF gives an example of how to create an snmpv3 user and run SNMP utilities to the user.

SNMP Traps:

Asynchronous notification from the agent to manager: This includes current sysUpTime value, an OID identifying the type of trap, and optional variable bindings. Destination addressing for traps is determined in an application-specific manner, typically through trap configuration variables in the MIB. The format of the trap message was changed in SNMPv2 and the PDU was renamed SNMPv2-Trap.

SNMP and its traps in 7-Mode and C-Mode Data ONTAP:

Standard SNMP traps:

There are 5 standard SNMP traps, as per RFC 1215:

• linkDown - This trap is generated on bringing down the active physical port, which is up (the ifAdminStatus should be changed from up to down/ The ifindex number information is not included in the trap message. )
• linkUp - This trap will be generated when you bring up the physical port which is down (the ifAdminStatus should be changed from down to up/The ifindex number information is not included in the trap message. )
• warmStart - A warmstart trap is generated when you do a normal reboot
• coldStart - A coldStart trap signifies that the sending protocol entity is reinitializing itself in such a way that the agent's configuration or the protocol entity implementation may be altered
• authenticationFailure - An authenticationFailure trap is generated when a user is trying to log in to the system using incorrect privileges

NetApp-built-in SNMP traps:

NetApp has a large number of built-in traps for the convenience of SNMP users. The file /mroot/etc/mib/netapp.mib has a list of the built-in traps. Each trap has a unique identifier or trap code. An example of a built-in trap is volumeOnline and its trap code is 276. The information below is taken from the netapp.mib file.

The OID is followed by the NOTIFICATION-TYPE tag, which indicates that it is a trap and is associated with the description and trap-code, 276 in this case.

volumeOnline                    NOTIFICATION-TYPE
  OBJECTS                       {productTrapData, productSerialNum}
  STATUS                          current
  DESCRIPTION              Volume is online now. The string sent with trap specifies name of volume which is online now.
        
         ::= { netapp 0 276 }

User-Defined SNMP traps:

These are traps that can be configured based on user requirements. Even though NetApp has some built-in traps, the user might still want to generate events for other reasons. 7-Mode has the infrastructure to support user-defined traps, but C-Mode infrastructure for Data ONTAP 8.1 does not support user-defined traps. These are the planned UIs (after Data ONTAP 8.1) that will help configure user-defined traps:

• snmp-trap-list
• snmp-trap-set
• snmp-trap-delete
• snmp-trap-reset
• snmp-trap-load

Tie-in w/ EMS:

Traps are tied to EMS events. SNMP events can be generated using the event CLI as well.
All the traphosts that are added to the SNMP traphost list will get replicated to another table called the event destination table under the traphost entry.
                               
test-01::*> system snmp traphost show
        TRAPHOST1
        TRAPHOST2
test-01::*> event destination show -name traphost

                  Name: traphost

      Mail Destination: -
      SNMP Destination: TRAPHOST1
                        TRAPHOST2
    Syslog Destination: -
       Syslog Facility: -
   SNMP Trap Community: public
Hide Parameter Values?: false

The following can be done using event * CLI:

A new SNMP host can be added using the event destination create CLI. When the host is added to the default traphost list, it gets replicated in the SNMP traphost list and all events that are triggered in the cluster will be sent out to this host.

csiqa-3170-6a1365754940::*> snmp traphost
        -

csiqa-3170-6a1365754940::*> event destination show -name traphost

                  Name: traphost
      Mail Destination: -
      SNMP Destination: -
    Syslog Destination: -
       Syslog Facility: -
   SNMP Trap Community: qwerty
Hide Parameter Values?: false

csiqa-3170-6a1365754940::*> snmp community

csiqa-3170-6a1365754940
        ro  qwerty

csiqa-3170-6a1365754940::*> event destination modify -name traphost -hide-parameters false -snmp 10.229.88.174 -snmp-community qwerty

csiqa-3170-6a1365754940::*> event destination show -name traphost                                                                   

                  Name: traphost
      Mail Destination: -
      SNMP Destination: csiqa-labopt-rh5-003.gdl.englab.netapp.com
    Syslog Destination: -
       Syslog Facility: -
   SNMP Trap Community: qwerty
Hide Parameter Values?: false

csiqa-3170-6a1365754940::*> snmp traphost
        csiqa-labopt-rh5-003.gdl.englab.netapp.com (csiqa-labopt-rh5-003.gdl.englab.netapp.com) <10.229.88.174>

csiqa-3170-6a1365754940::*>

However, if the user wants to configure a host to receive only particular events, the event route CLI can be used to route any event to that destination. To list the events, do the following:

csiqa-3070-591287556400::*> event route show
                                                               Freq    Time
Message                          Severity       Destinations   Threshd Threshd
-------------------------------- -------------- -------------- ------- -------
EthrOutput.FamilyType.Err        ERROR          -              0       0
LUN.clone_snapshot_destroyed     NOTICE         -              0       0
LUN.destroy                      INFORMATIONAL  -              0       0
LUN.space_reservation_not_honored
                                 NOTICE         -              0       0
LUN.volume_processing_failed_no_space
                                 ERROR          -              0       0
Nblade.DidNotInitialize          ERROR          -              0       0
Nblade.JunctionRootLookup        WARNING        -              0       0
Nblade.Nfs4IllegalDirentName     ERROR          -              0       0
Nblade.NfsRaidError              ERROR          -              0       0
Map the corresponding event to the Destination required.
Not all events are SNMP trap enabled. In order to know all the traps associated with snmp traps, use the following CLI :
csiqa-3070-591287556400::*> event route show -snmp-support true
                                                               Freq    Time
Message                          Severity       Destinations   Threshd Threshd
-------------------------------- -------------- -------------- ------- -------
app.log.alert                    ALERT          -              0       0
app.log.crit                     CRITICAL       -              0       0
app.log.debug                    DEBUG          -              0       0
app.log.emerg                    EMERGENCY      -              0       0
app.log.err                      ERROR          -              0       0
app.log.info                     INFORMATIONAL  -              0       0
app.log.notice                   NOTICE         -              0       0
app.log.warn                     WARNING        -              0       0
asup.general.create              ERROR          -              0       0
asup.general.drop                ERROR          -              0       0
asup.general.drop.enqueue        INFORMATIONAL  -              0       0

SNMP traps - How to configure traps and generate events

Add the traphosts using the snmp traphost CLI:

$> snmp traphost add
-OR-
$> snmp traphost add

Note: Ensure that dns is configured on the cluster in order to resolve the traphost names. 

1. Ensure that SNMP protocol is enabled on the filer:
   
   ::> options -option-name snmp.enable on

2. Enable SNMP traps on the cluster. This can be done in either of the following ways:       
                
   Run the following command:
   Cluster::> snmp init 1
   -OR-
   Run the ZAPI :
   $> ontapi snmp-trap-enable

3. Trigger and monitor events.

traps.dat info - reference: Burt # 460968

traps.dat file is particularly for user-defined-traps. UDT support is not in Data ONTAP C-Mode as yet (as of Data ONTAP 8.1.1)

traps.dat file has the extra 1 at the end to help differentiate between OIDs inside of tables and OIDs that end in 0 (outside of tables). OIDs inside of tables should not be trapped against, so the extra obfuscation actually helps to reduce invalid user-defined traps 

C-Mode OID info - reference: Burt # 460968 

The OIDs that are seen in the netapp.mib file are the keys and are not the actual absolute OID values. The best way to figure this out is by actually performing snmpwalk by dropping the last digit:

Example: 

• [rakeshc@cyclnb01 ~/p4]$ snmpwalk -v 1 -c public 10.10.10.10 .1.3.6.1.4.1.789.1.5.11.1.2
• SNMPv2-SMI::enterprises.789.1.5.11.1.2.1026 = STRING: "aggr0"
• SNMPv2-SMI::enterprises.789.1.5.11.1.2.1030 = STRING: "aggr0_br3040n2_rtp"
• SNMPv2-SMI::enterprises.789.1.5.11.1.2.1034 = STRING: "n1_aggr1"
• SNMPv2-SMI::enterprises.789.1.5.11.1.2.1038 = STRING: "n2_aggr1"
• SNMPv2-SMI::enterprises.789.1.5.11.1.2.1050 = STRING: "coral_aggr" 

It gives 5 different OIDs above (1026, 1030, 1034, 1038, 1050) and then perform the following:

• [rakeshc@cyclnb01 ~/p4]$ snmpwalk -v 1 -c public 10.61.76.140 .1.3.6.1.4.1.789.1.5.11.1.2.1026
• SNMPv2-SMI::enterprises.789.1.5.11.1.2.1026 = STRING: "aggr0"

The detailed procedure of how to generate different types of traps/events is shown in the Appendix section below:

Enable SNMP and run snmp utilities:

Running snmpwalk/snmpget/snpgetnext etc.. on a cluster and also the config steps to do this

 DOT_cluster::*> options snmp.enable

DOT_cluster
    snmp.enable                       on

DOT_cluster::*> network interface show -vserver DOT_cluster
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
DOT_cluster
            cluster_mgmt up/up    10.238.44.38/18    node1         e0c     true

DOT_cluster::*>


bash-3.2$ snmpwalk -c public -v 1 10.238.44.38 .1.3.6.1.4.1.789.1.5.11.1.2
SNMPv2-SMI::enterprises.789.1.5.11.1.2.1026 = STRING: "aggr0"
SNMPv2-SMI::enterprises.789.1.5.11.1.2.1030 = STRING: "aggr0_partnernode"
SNMPv2-SMI::enterprises.789.1.5.11.1.2.1034 = STRING: "aggr_node1"
bash-3.2$

Additional Information

Add your text here.