Autosupport fails delivery with error: Operation timed out in ONTAP
Applies to
- ONTAP 9
- AutoSupport
- HTTPS
- SMTP
- Transparent Proxy / Firewall
Issue
One or more nodes in the cluster fail to send ASUP (AutoSupport) data due to a timeout reaching the ASUP destination. See below for examples of various commands that may report this error:
- Command
system node autosupport check show-details
reportsError: Timeout was reached when connecting to the server - 'support.netapp.com'
Example:
cluster1::> system node autosupport check show-details -node cluster1-1a
Node: cluster1-1a
Category: https
Component: https-put-destination
Status: failed
Detail: HTTPS PUT connectivity check failed for destination:
https://support.netapp.com/put/AsupPut/. Error: Timeout
was reached when connecting to the server -
'support.netapp.com'
Corrective Action: Possible network configuration issue. Ensure that you
have network connectivity Component: https-post-destination
Status: failed
Detail: HTTPS POST connectivity check failed for destination:
https://support.netapp.com/asupprod/post/1.0/postAsup.
Error: Timeout was reached when connecting to the server
- 'support.netapp.com'
Corrective Action: Possible network configuration issue. Ensure that you
have network connectivity
...
Category: on-demand
Component: ondemand-server
Status: failed
Detail: AutoSupport OnDemand connectivity check failed for
destination:
https://support.netapp.com/aods/asupmessage. Error:
Timeout was reached when connecting to the server -
'support.netapp.com'
Corrective Action: Possible network configuration issue. Ensure that you
have network connectivity
Category: configuration
Component: configuration
Status: ok
Detail: No confhistory iguration issues found.
5 entries were displayed.
- Command
system autosupport history show
reportstransmission-failed Failed to connect to support.netapp.com ... Operation timed out
Example:
cluster1::*> system autosupport history show -node * -fields node,seq-num,status,error,destination
node seq-num destination status error
---------- ------- ----------- ------ -----
node1-01 762159 smtp ignore
node1-01 762159 http transmission-failed Failed to connect to support.netapp.com port 443: Operation timed out
- In
notifyd.log:
errormessage: Closing connection #0
followed by HTTP/SMTP errors:
Example (HTTPS):
(message: Trying 216.240.21.18...
(message: Failed to connect to support.netapp.com port 443: Operation timed out)
Example (SMTP):
(message: Trying MAIL_HOST_IP...
(message: Operation timed out)
Cause
- "Operation timed out" means there is a networking issue from the node to support.netapp.com or the SMTP server
- The AutoSupport transmission was able to route out of the node, but either could not reach the destination (support.netapp.com for HTTPS), or did not receive a response back. This can be related to:
- Routing configuration
- Firewall configuration
- Transparent Proxy configuration
- Incorrect ONTAP cluster configuration
Solution
- Work with your networking team to confirm:
- Any firewall/transparent proxy should allow connections from ONTAP (e.g. ACLs)
- Allowlist all cluster and node management LIF IP addresses
- Ensure that any firewall is configured to allow outbound access to support.netapp.com for the appropriate port:
- HTTPS: 443
- SMTPS: 587
- SMTP: 25
- Any firewall/transparent proxy should allow connections from ONTAP (e.g. ACLs)
Other less common solutions:
- Ensure the route used in ONTAP can reach the destination (support.netapp.com or SMTP server)
- Verify there are no duplicate IP addresses in the network that could inhibit AutoSupport communication
- Ensure that e0m (if configured) is on the same subnet as mgmt lifs
- Confirm the routing table is configured correctly in ONTAP, and that the node routing table has a default gateway for the subnet used in the mgmt LIF.
Additional Information
Notes:
- AutoSupport transmission can occur over both node management and cluster management LIFs
- When intercluster and management LIFs are configured in the same subnet or the intercluster gateway has a lower Metric weight to associate with a static route and if the route associates with an intercluster LIF, the management traffic will attempt to pass traffic through the intercluster LIF
- Set the intercluster LIF and management LIF in different subnets to avoid this issue
- Based on your Network routing this may be blocked by an external firewall and the AutoSupport and NTP connections will fail
- Temporarily test this by running the
network interface modify -vserver vserver name -lif intercluster LIF -status-admin down
to verify traffic uses the mgmt LIF - Once confirmed, online the LIF again
network interface modify -vserver vserver name -lif intercluster LIF -status-admin up
- Temporarily test this by running the
- To reach
notifyd.log -
::> node run -node <node_name> -command rdfile /etc/log/mlog/notifyd.log