Increased latency and timeouts due to vscan latency
Applies to
- ONTAP 9
- VSCAN
Issue
- Accessing any file is very slow
- Increased latency is seen when copying files
-
EMS logs show Nblade_CifsOperationTimedOut error :
kernel: Nblade_CifsOperationTimedOut_1:error]: params: {'remoteBladeID': '<id> (<node_name>)', 'cmdRestartCnt': '0', 'vserverId': '7', 'commandName': 'SMB2_COM_CREATE', 'isQosEnabled': 'QoS_disabled', 'lastSpinNpError': 'SPINNP_ERR_OFFBOX_VSCAN_REQD', 'localIpAddress': '10.xx.xx.xx', 'lastCsmError': 'CSM_OK', 'clientIpAddress': '10.xx.xx.xx', 'vserverName': '<svm_name>', 'suspensionCnt': '<count>', 'dsId': '<id>'}Tue May 21 10:36:18 +0900 [node-01: kernel: ems.engine.suppressed:debug]: Event 'Nblade.CifsOperationTimedOut' suppressed 60 times in last 273 seconds.-
The performance archive shows high latency on the VSCAN server:
Cluster: cluster1 (4cfa91c5-761a-xxxxx-xxxxx-xxxxxxx)
Node: node1 (4cfa91c5-761a-xxxxx-xxxxx-xxxxxxx)
Time Range: 2024-07-21 18:00:00.999+00:00 - 2024-07-22 00:00:00.001+00:00
time interval offbox_vscan_server offbox_vscan_server offbox_vscan_server
instance scan_latency scan_latency_base
(us)
------------------------------------ ----------------------------------- ------------------- -------------------
2024-07-21 18:31:18 - 18:36:18+00:00 svm1:10.x.x.1:node1 129298.60 5
2024-07-21 18:36:18 - 18:41:18+00:00 svm1:10.x.x.1:node1 121461.00 3
2024-07-21 18:41:18 - 18:46:18+00:00 svm1:10.x.x.1:node1 152285.50
2024-07-21 18:56:18 - 19:01:18+00:00 svm1:10.x.x.1:node1 243688.50
2024-07-21 19:01:18 - 19:06:18+00:00 svm1:10.x.x.1:node1 314138.00
2024-07-21 19:11:18 - 19:16:18+00:00 svm1:10.x.x.1:node1 141676.00
2024-07-21 19:16:18 - 19:21:18+00:00 svm1:10.x.x.1:node1 143551.20 5
2024-07-21 19:21:18 - 19:26:18+00:00 svm1:10.x.x.1:node1 138153.50
2024-07-21 19:26:18 - 19:31:18+00:00 svm1:10.x.x.1:node1 366887.00 3
2024-07-21 19:31:18 - 19:36:18+00:00 svm1:10.x.x.1:node1 1776264.29 7
2024-07-21 19:36:18 - 19:41:18+00:00 svm1:10.x.x.1:node1 390445.80 5
2024-07-21 19:41:18 - 19:46:18+00:00 svm1:10.x.x.1:node1 2185337.75 4
2024-07-21 19:46:18 - 19:51:18+00:00 svm1:10.x.x.1:node1 342919.09 11
2024-07-21 19:51:18 - 19:56:18+00:00 svm1:10.x.x.1:node1 566124.60 10