How to capture packet traces (tcpdump) on ONTAP 9.2 to 9.9 systems

Applies to

ONTAP 9.2 to 9.9

Description

Procedure to capture packet traces (tcpdump) on ONTAP 9.2 till 9.9 systems.

Procedure

 

Warning Always filter packet traces on a single client IP whenever possible If multiple clients are affected, select 1 to focus on for troubleshooting Use the -buffer flag on systems with a minimum value of 4096 to ensure the trace doesn't filter packets Don't follow this procedure to capture CRC errors being received in a network port.

 

• This will start a packet trace (network tcpdump start (netapp.com)):

::> network tcpdump start -node <node> -port <port/ifgrp/vlan> -address <client-ip/lif-ip/etc.> -buffer-size 2097151

Notes:  

• -node and -port are mandatory
• Wildcards cannot be used for the node or port for this command
• if an ifgrp (e.g. a0a) has VLANs (e.g. a0a-117), specifying a0a will only capture Native VLAN (untagged) traffic
• The -address option can specify only one IP address to filter the trace.
• The -protocol-port option allows for the trace to be filtered by one port for both TCP and UDP traffic.

• Rolling trace:

::> network tcpdump start -node cl1-n1 -port e0b -file-size 512 -rolling-traces 4 -address 10.1.1.2 -protocol-port 445

Note:

• This trace rolls up to 4 trace files of size 512 MB each (oldest file removed first).
• It traces on the selected port, filtering for IP address 10.1.1.2 and TCP/UDP port 445.]
   
• To show running packet traces (network tcpdump show (netapp.com)):

::> network tcpdump show

• To stop a specific packet trace  (network tcpdump stop (netapp.com)):

 ::> tcpdump stop -node <node> -port [*|<port>]

• To stop all traces:

::> tcpdump stop *

• To show packet trace files (network tcpdump trace show (netapp.com))

::> network tcpdump trace show

Note: ONTAP stores trace files in /mroot/etc/log/packet_traces

• Deleting packet traces on a node (network tcpdump trace delete (netapp.com))

::> network tcpdump trace delete -node <node> -trace-file *

• Retrieve packet traces via web browser:
  • http(s)://<CLUSTER_MGMT_IP>/spi/<NODE_NAME>/etc/log/packet_traces/

Note: Cluster credentials are needed to access the SPI

Space considerations

• If collecting large traces, use the df -h command to confirm the node's root volume has enough space.
  • More than twice the total trace size (file size times number of traces) should be available before starting packet traces.
• By default, trace files are added to snapshot copies, thus vol0 (root volume) may fill up quickly causing an outage
  • To avoid consuming root volume space with trace files captured in snapshots:
    • Disable automatic Snapshots on the node root volume from nodeshell of the node where the trace is being collected

::> run -node <node> -command "vol options vol0 nosnap on" 

• Freeing up space on a node’s root volume
  • Delete Snapshots created during a trace

::> run -node <node> -command "snap list vol0"

::> run -node <node> -command "snap delete vol0 <snap name>"

• After packet trace collection is finished, re-enable root volume Snapshots if they were originally enabled
  • From nodeshell of the node where Snapshots were disabled

::> run -node <node> -command "vol options vol0 nosnap off"