Deletion takes longer for large directories when the SMB CLOSE requests are runnig for longer duration

Applies to

ONTAP 9

SMB

Issue

• It takes a long time to delete a long directory, the long-running SMB CLOSE requests(WAFL_SPINNP_CLOSE) is the major contributor to the performance drop, and it's caused by the extremely long cpu run time in WAFL cpu scheduling.
• This could be confirmed in a number of different places. 
  • High Service Response Time for SMB CLOSE requests in the packet trace
  • High latency for Close operations in Object spinhi
    • spinhi:<volume_name>:spinhi_fileop_latency.Close:41740.05u
  • High latency for Close operations in Object smb or smb2
    • smb2:<svm_name>:close_latency:39380.24us
  • High CPU wait time(Per-Msg Wait Avg) in waffinity_stats, along with a very high CPU run time(Per_msg Wait Avg)

Class Queue     Msg Type                            #Msgs (new+restarts)    Per-Msg CPU Avg             Per-Msg Wait Avg

SERIAL hipri    WAFL_SPINNP_CLOSE                   62419                   1343                        44103

Note: The average CPU run time is 1343us, while the Average CPU wait time is 44103us in this example

• Directory blocks take the majority of all the blocks read by Readahead Engine

ra.requested                           =  230178001

 ra.dir_blocks                          =  223951083