CIFS client connections to share hang with error Nblade_CifsOperationTimedOut_1 for commandName SMB2_COM_SESSION_SETUP
Applies to
- ONTAP 9
- CIFS
Issue
- CIFS client connections to share hang
- EMS logs show:
[node-01: kernel: Nblade_CifsOperationTimedOut_1:error]: params: {'remoteBladeID': '00000000-0000-0000-0000-000000000000', 'cmdRestartCnt': '0', 'vserverId': '27', 'commandName': 'SMB2_COM_SESSION_SETUP', 'isQosEnabled': 'QoS_disabled', 'lastSpinNpError': 'SPINNP_NO_FO_ERROR', 'localIpAddress': '10.xx.yy.22', 'lastCsmError': 'CSM_OK', 'clientIpAddress': '10.xx.zz.88', 'suspensionCnt': '1', 'dsId': '0'}
secd: secd.rpc.server.request.dropped:debug]: The RPC secd_rpc_auth_extended sent from NBLADE_CIFS was dropped by SecD due to memory pressure. in Secd log: SecD RPC Server: Too many outstanding Generic RPC requests: sending System Error to RPC RPC TOOK TOO LONG: | RPC used 6617 seconds (max is 23)
- SECD logs show
RPC requests sitting in SECD queue for long times.
[kern_secd:info:16176] | [000.000.015] debug: Worker Thread 34507281152 processing RPC 151:secd_rpc_auth_extended with request ID:30219 which sat in the queue for 21 seconds. { in run() at src/server/secd_rpc_server.cpp:2306 }
[kern_secd:info:16176] | [000.000.013] debug: Worker Thread 34507279872 processing RPC 151:secd_rpc_auth_extended with request ID:30184 which sat in the queue for 19 seconds. { in run() at src/server/secd_rpc_server.cpp:2306 }
- SVMs on the same node as the affected LIF (localIpAddress in EMS message) may also be impacted
- Packet traces shows huge number of authentication
CIFS Session Setuprequests in a short duration and some requests receivingError: STATUS_INTERNAL_ERROR
347672 2021-08-12 08:01:55.550 10.xx.yy.224 10.xx.zz.22 SMB2 Session Setup Request
352783 2021-08-12 08:02:36.293 10.xx.zz.22 10.xx.yy.224 SMB2 Session Setup Response, Error: STATUS_INTERNAL_ERROR
- Clients with high number of multiple sessions send many authentication requests as per packet trace analysis:
::> cifs session show -vserver <svm>
::> cifs session file show -vserver <svm> -connection-id <integer>
