CIFS inaccessible after Snapmirror Cloud Restore
Applies to
- Cloud Volumes ONTAP 9.10.1
- CIFS
- Snapmirror Cloud restore
Issue
- After a Snapmirror cloud restore, CIFS shares become inaccessible, with operations timing out.
- Event log errors similar to:
Mon Feb 28 12:11:50 1100 [node01: kernel: Nblade_CifsOperationTimedOut_1:error]: params: {'remoteBladeID': '3855f5a6-7d64-11ec-b35a-4d5fcf2e348d (node01)', 'cmdRestartCnt': '0', 'vserverId': '3', 'commandName': 'SMB2_COM_CREATE', 'isQosEnabled': 'QoS_disabled', 'lastSpinNpError': 'SPINNP_NO_FO_ERROR', 'localIpAddress': '10.10.100.143', 'lastCsmError': 'CSM_BACKPRESSURE', 'clientIpAddress': '10.10.255.121', 'vserverName': 'svmtest', 'suspensionCnt': '2069', 'dsId': '1037'}
Mon Feb 28 12:16:51 1100 [node-01: kernel: ems.engine.suppressed:debug]: Event 'Nblade.CifsOperationTimedOut' suppressed 51 times in last 293 seconds.
Mon Feb 28 12:26:57 1100 [node01: kernel: Nblade_CifsOperationTimedOut_1:error]: params: {'remoteBladeID': '3855f5a6-7d64-11ec-b35a-4d5fcf2e348d (node01)', 'cmdRestartCnt': '0', 'vserverId': '3', 'commandName': 'SMB2_COM_TREE_CONNECT', 'isQosEnabled': 'QoS_disabled', 'lastSpinNpError': 'SPINNP_NO_FO_ERROR', 'localIpAddress': '10.10.100.143', 'lastCsmError': 'CSM_BACKPRESSURE', 'clientIpAddress': '10.10.255.126', 'vserverName': 'svmtest', 'suspensionCnt': '2065', 'dsId': '1031'}- High latency ~40s for SMB2 create and tree-connect requests
cluster::*> statistics show-periodic -object smb2 -counter create_latency|tree_connect_latency -instance <svm> -summary false -interval 10
cluster: smb2.svmtest: 3/15/2022 14:36:37
tree
create connect
latency latency
-------- --------
42332159us 44us
42332191us 42us
42331286us 42335102us
42332996us 0us
- From packet trace, SMB2 Tree Connect and Create Requests fail with STATUS_INTERNAL_ERROR after 40s.
No. delta src dst PROT TCP_Len Response_time Info 135676 0.061154000 10.10.100.143 10.10.168.244 SMB2 77 [TCP ACKed unseensegment] Create Response, Error: STATUS_INTERNAL_ERROR 135680 0.012470000 10.10.168.244 10.10.100.143 SMB2 648 Create Request File: FASHION-2022-006_22_14_08_09.pdf 155007 0.001199000 10.10.100.143 10.10.168.242 SMB2 77 40.014543000 Create Response,Error: STATUS_INTERNAL_ERROR 155012 0.006675000 10.10.168.242 10.10.100.143 SMB2 568 Create Request File: FILES_Output.xml 182225 0.000102000 10.10.100.143 10.10.168.246 SMB2 77 40.014702000 Tree Connect Response, Error: STATUS_INTERNAL_ERROR 182238 0.001300000 10.10.168.246 10.10.100.143 SMB2 120 Tree Connect Request Tree: \10.10.100.143group7$ 185087 0.042234000 10.10.100.143 10.10.168.244 SMB2 77 Create Response, Error: STATUS_INTERNAL_ERROR 185088 0.007147000 10.10.168.244 10.10.100.143 SMB2 284 Create Request File: NShareTele 195602 0.002420000 10.10.100.143 10.10.168.248 SMB2 77 Tree Connect Response, Error: STATUS_INTERNAL_ERROR 195604 0.005818000 10.10.168.248 10.10.100.143 SMB2 120 Tree Connect Request Tree: \10.10.100.143group7$ 220790 0.088176000 10.10.100.143 10.10.168.241 SMB2 77 40.010021000 Tree ConnectResponse, Error: STATUS_INTERNAL_ERROR 220791 0.005019000 10.10.168.241 10.10.100.143 SMB2 120 Tree Connect Request Tree: \10.10.100.143group9$ 226800 0.019917000 10.10.100.143 10.10.168.244 SMB2 77 40.005785000 Create Response, Error: STATUS_INTERNAL_ERROR 226803 0.052802000 10.10.168.244 10.10.100.143 SMB2 648 Create Request File: FFASHION-2022-006_22_14_08_09.pdf 270263 0.118639000 10.10.100.143 10.10.168.242 SMB2 77 40.002190000 Create Response, Error: STATUS_INTERNAL_ERROR