Skip to main content
NetApp Knowledge Base

Qtree creation via API is intermittently failing

  1. Last updated
  2. Save as PDF
Views:
5
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
nas
Last Updated:

Applies to

ONTAP 9

Issue

  • Audit logs shows pending & success but not Qtree Qtree_X does not exist in CLI.

Fri Jan 16 2026 09:59:43 +02:00:: yok-cdot17: http :: 10.48.149.214:38988 :: Node1:napi_admin_user :: POST /api/storage/qtrees/?return_records=true : {"security_style": "unix", "volume": {"name": "swgwork10"}, "name": "Qtree_X", "unix_permissions": "700", "group": {"id": 30}, "user": {"id": 152382}, "svm": {"name": "Node1"}, "export_policy": {"name": "yok-lab-default"}} :: PendingFri Jan 16 2026 10:00:07 +02:00:: yok-cdot17: http :: 10.48.149.214:38988 :: Node1:napi_admin_user :: POST /api/storage/qtrees/?return_records=true : {"security_style": "unix", "volume": {"name": "swgwork10"}, "name": "Qtree_X", "unix_permissions": "700", "group": {"id": 30}, "user": {"id": 152382}, "svm": {"name": "Node1"}, "export_policy": {"name": "yok-lab-default"}} :: Success:Fri Jan 16 2026 16:31:05 +02:00:: yok-cdot17: http :: 10.48.149.214:24478 :: Node1:napi_admin_user :: POST /api/storage/qtrees/?return_records=true : {"security_style": "unix", "volume": {"name": "swgwork10"}, "name": "Qtree_X", "unix_permissions": "700", "group": {"id": 30}, "user": {"id": 152382}, "svm": {"name": "Node1"}, "export_policy": {"name": "yok-lab-default"}} :: PendingFri Jan 16 2026 16:31:20 +02:00:: yok-cdot17: http :: 10.48.149.214:24478 :: Node1:napi_admin_user :: POST /api/storage/qtrees/?return_records=true : {"security_style": "unix", "volume": {"name": "swgwork10"}, "name": "Qtree_X", "unix_permissions": "700", "group": {"id": 30}, "user": {"id": 152382}, "svm": {"name": "Node1"}, "export_policy": {"name": "yok-lab-default"}} :: Success:

  • Qtree Create   PENDING : 09:59:43
  • Qtree Create   SUCCESS : 10:00:07
  • The delay between Pending --> Success was 24 seconds.
  • This is very long for a qtree creation. Normally this takes milliseconds.
  • CIFS Share Create - PENDING : 10:00:08 (one second after qtree success)
  • CIFS Share Create - ERROR 10:00:08 (same second)
  • Even though ONTAP returned Success for the qtree create, the filesystem path did NOT exist yet.
  • This means:
    • Config-layer object was created
    • Namespace-layer instantiation was NOT complete
    • This is why ONTAP errors out: The specified path "/vol/swgwork10/Qtree_X" does not exist
    • Typical qtree creation time: Normal systems: 10 200 ms
    • Slight load: 200 500 ms
  • From the event logs, during the qtree creation window (09:59:43   10:00:07), and immediately after, the node emitted EMS events showing:
    • Severe CIFS authentication failures
    • TLS handshake timeouts
    • CSM (Cluster Session Manager) connection aborts
    • High CP counts (WAFL under pressure)
    • Quota warnings
    • SSH CPS limit exceeded (too many incoming connections)
  • SECD lookups failing for UNIX UIDs

Fri Jan 16 10:04:34 +0200 [Node2: secd: secd.cifsAuth.problem:error]: vserver (Node1) General CIFS authentication problem. Error: User authentication procedure failed CIFS SMB2 Share mapping - Client Ip = 10.7.1.52 [ 0 ms] Login attempt by domain user 'L-WIN-SYMBOLS\Administrator' using NTLMv2 style security [ 0] Using a cached connection to ykdc195.nvidia.com [ 2] Authentication failed with DC YKDC195. Not retriable. (Status: NT_STATUS_AUTH_NO_SUCH_USER(0xc0000064)) [ 2] Login attempt by local user 'L-WIN-SYMBOLS\Administrator' using NTLMv2 style security [ 2] 'L-WIN-SYMBOLS' is not a valid domain. Trying login as local user 'YOK-C17-LAB01\Administrator'. **[ 2] FAILURE: CIFS authentication failed
Fri Jan 16 10:05:06 +0200 [Node2: ktlsd: ktls.failed:notice]: "The TLS connections have failed several times with remote host '10.19.136.109' in IPspace '4294967295', for which the latest reason given is: TLS: Handshake timed out."
Fri Jan 16 10:05:22 +0200 [Node2: secd: secd.authsys.lookup.failed:error]: Unable to retrieve credentials for UNIX user with UID (988) on Vserver (yok-cdot17-labhome01) for client with IP address (10.7.177.7).
Fri Jan 16 10:06:17 +0200 [Node2: CsmMpAgentThread: csm.createSessionFailed:debug]: Cluster Session Manager (CSM) failed to create session (req=Node2:dblade, rsp=mtv-cdot01-01:dblade, uniquifier=1106487ccd45fcef) with transport type UNASSIGNED, session tag WAFL_REMOTE, record state STARTING, CSM error CSM_CONNABORTED, low-level error UNKNOWN, socket error 0, and TLS error 0.
Fri Jan 16 10:06:44 +0200 [Node2: config_thread: raid.aggr.log.CP.count:debug]: params: {'aggregate_type': 'Aggregate', 'aggregate_name': 'yok_cdot17_02_aggr0', 'aggregate_uuid': '577af47d-99c8-4480-91e2-37deb0ce6aa3', 'home_owner_id': '538343687', 'home_owner_name': 'Node2', 'CP_count': '3262733'}
Fri Jan 16 10:07:10 +0200 [Node2: secd: secd.authsys.lookup.failed:error]: Unable to retrieve credentials for UNIX user with UID (985) on Vserver (yok-cdot17-labhome01) for client with IP address (10.7.73.53).
Fri Jan 16 10:07:24 +0200 [Node2: mgwd: cpeer.psk.unknown.cluster:error]: ONTAP received a TLS request to authenticate with a remote cluster's pre-shared key, but the remote cluster UUID (10842289-08e5-11ee-82f1-d039ea548056) is unrecognized. The probe arrived from address 10.5.1.193 in IPspace Default.
Fri Jan 16 10:07:38 +0200 [Node2: wafl_exempt22: wafl.quota.qtree.exceeded:notice]: tid 1104: tree quota exceeded on volume labhome2@vserver:752e7266-19c0-11f0-9e27-d039eac07550. Additional warnings will be suppressed for approximately 60 minutes or until a 'quota resize' is performed.
Fri Jan 16 10:07:38 +0200 [Node2: wafl_exempt22: quota.exceeded:debug]: params: {'ltype': 'hard', 'volname': 'labhome2', 'app': '', 'volident': '@vserver:752e7266-19c0-11f0-9e27-d039eac07550', 'limit_item': 'disk', 'limit_value': '10485760', 'user': '', 'qtree': ', treeid=1104', 'vfiler': ''}
Fri Jan 16 10:07:39 +0200 [Node2: secd: secd.cifsAuth.problem:error]: vserver (Node1) General CIFS authentication problem. Error: User authentication procedure failed CIFS SMB2 Share mapping - Client Ip = 10.7.1.52 [ 0 ms] Login attempt by domain user 'L-WIN-SYMBOLS\Administrator' using NTLMv2 style security [ 0] Using a cached connection to ykdc195.nvidia.com [ 2] Authentication failed with DC YKDC195. Not retriable. (Status: NT_STATUS_AUTH_NO_SUCH_USER(0xc0000064)) [ 2] Login attempt by local user 'L-WIN-SYMBOLS\Administrator' using NTLMv2 style security [ 2] 'L-WIN-SYMBOLS' is not a valid domain. Trying login as local user 'YOK-C17-LAB01\Administrator'. **[ 2] FAILURE: CIFS authentication failed
Fri Jan 16 10:08:50 +0200 [Node2: config_thread: raid.aggr.log.CP.count:debug]: params: {'aggregate_type': 'Aggregate', 'aggregate_name': 'yok_cdot17_02_qlc_aggr1', 'aggregate_uuid': 'f7cf59b6-c81c-11ef-92cd-d039eac190f6', 'home_owner_id': '538343687', 'home_owner_name': 'Node2', 'CP_count': '3245495'}
Fri Jan 16 10:10:12 +0200 [Node2: emsgenr: xinetd.hit.cps.limit:error]: Number of incoming network connections exceeded the configured limit of 10 connections per second for the service ssh. This service will be stopped and restarted after 60 seconds.
Fri Jan 16 10:10:13 +0200 [Node2: secd: secd.cifsAuth.problem:error]: vserver (yok-cdot17-labhome01) General CIFS authentication problem. Error: User authentication procedure failed CIFS SMB2 Share mapping - Client Ip = 10.127.21.212 **[ 0] FAILURE: CIFS authentication failed
Fri Jan 16 10:10:16 +0200 [Node2: client_common_RPC: ems.engine.suppressed:debug]: Event 'csm.rpcProblem' suppressed 3 times in last 600 seconds.
Fri Jan 16 10:10:16 +0200 [Node2: client_common_RPC: csm.rpcProblem:debug]: params: {'appName': 'VIF Manager', 'filename': 'src/Csm/BladeLocationService.cc', 'lineno': '996'}
Fri Jan 16 10:10:17 +0200 [Node2: sshd: sshd.auth.loginDenied:notice]: params: {'message': 'Failed password for root from 10.127.21.212 port 42780 ssh2 '}
Fri Jan 16 10:10:17 +0200 [Node2: sshd: sshd.auth.loginDenied:notice]: params: {'message': 'Failed password for invalid user admin from pii_encrypt/22kZmUz1EOkhkq7ZVlBGbMmHRW3RSWtNj7z6b4Izg0c=/pii_encrypt port 42538 ssh2 '}

  • These all point to node resource contention, especially in:
    •  SECD
    • WAFL (metadata + namespace)
    • Networking (TLS/CSM)
    • Management interfaces (xinetd, SSH)

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.