How to capture packet traces (tcpdump) on ONTAP 9.2 to 9.9 systems

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 63,401

Visibility:: Public

Votes:: 37

Category:: ontap-9

Specialty:: nas

Last Updated:

Applies to

ONTAP 9.2 to 9.9

Description

Procedure to capture packet traces (tcpdump) on ONTAP 9.2 till 9.9 systems.

Procedure

Always filter packet traces on a single client IP whenever possible
If multiple clients are affected, select 1 to focus on for troubleshooting
Use the -buffer flag on systems with a minimum value of 4096 to ensure the trace doesn't filter packets
Don't follow this procedure to capture CRC errors being received in a network port.

simple trace:

::> network tcpdump start -node <node> -port <port-or-ifgrp> -address <ip-to-filter-on> -buffer-size 2097151

Note: ip-to-filter-on may be a client address or a lif on the selected port

This will start a packet trace on the specified port on the specified node (wildcards cannot be used for the node or port for this command, and file size of 1 GB).

To stop a packet trace: ::> tcpdump stop -node <node> -port [*|<port>]
A simple command to stop all traces is ::> tcpdump stop *
To show packet trace files:

::> network tcpdump trace show

Packet traces are stored in the following path:

/mroot/etc/log/packet_traces

Deleting an old packet trace

::> network tcpdump trace delete ? [-node] <nodename> Node Name [-trace-file] <text> Trace File

Mandatory fields

-node
-port must be a single physical (example e0g) or virtual port (example a0a-16)
- NOTE: choosing -port a0a will only capture traffic which is not vlan tagged
- If a lif is on a vlan, capture traffic on the vlan hosting the lif by specifying the vlan tag number (example -port a0a-16)

Options

::> network tcpdump start -node <node> -port <port> ? [[-address] <IP Address>] IP Address [ -protocol-port {1..65535} ] Protocol Port Number [ -file-size {1..65536} ] Trace File Size in MB [ -rolling-traces {1..64} ] Number of Rolling Trace Files

The -port field is mandatory
The -address option can specify only one IP address to filter the trace.
The -protocol-port option allows for the trace to be filtered by one port for both TCP and UDP traffic.
The -file-size option allows for modification of the trace file size from its default (1024 MB).
The -rolling-traces option specifies the number of traces files to save if using rolling packet traces.
- Note: If -rolling-traces is not used, a rolling trace with 2 files will be used.

Ensure that node's root volume has enough space if you need to collect large trace files, you can use the 'df -h' command to check it
- More than twice the total trace size (file size times number of traces) should be available before starting packet traces.
Be aware that, by default, the trace files will be added to snapshot copies and that vol0 (root volume) may fill up very quickly causing an outage
To avoid consuming root volume space with trace files captured in snapshots, use one of these two options
- Disable automatic Snapshots on the node root volume from nodeshell of the node where the trace is being collected
  - ::> run -node <node> -command "vol options vol0 nosnap on"
    - Automatic Snapshot copies is disabled. You may consider to delete old snapshots for vol0 based on your space requirements.
- Delete Snapshots created during a trace
  - ::> run -node <node> -command "snap list vol0"
  - ::> run -node <node> -command "snap delete vol0 <snap name>"
    - Reference, Freeing up space on a node’s root volume.
After packet-trace collection is finished, re-enable root volume Snapshots if they were originally enabled
- From nodeshell of the node where Snapshots were disabled
  - ::> run -node <node> -command "vol options vol0 nosnap off"

Rolling trace example

::> network tcpdump start -node <node> -port <port-or-ifgrp> -file-size 512 -rolling-traces 4 -address 10.1.1.2 -protocol-port 445

This trace rolls up to 4 trace files of size 512 MB each (oldest file removed first).
It traces on the selected port, filtering for IP address 10.1.1.2 and TCP/UDP port 445.]

Retrieving packet traces

The packet traces can be downloaded from the following location using a web browser of your choice:

http(s)://<CLUSTER_MGMT_IP>/spi/<NODE_NAME>/etc/log/packet_traces/

Cluster credentials are needed to access the SPI