Skip to main content
NetApp Knowledge Base

High write latency in Azure CVO

  1. Last updated
  2. Save as PDF
Views:
507
Visibility:
Public
Votes:
0
Category:
cloud-volumes-ontap-cvo
Specialty:
cloud_performance
Last Updated:

Applies to

  • ONTAP 9
  • Azure
  • CVO (Cloud Volume ONTAP)

Issue

High write latency in Azure CVO (Cloud Volume ONTAP) caused by slow NVRAM transfer when using page-blob disks

Cause

  • The communication with the page-blob disks is done using API calls over HTTP/S
  • If that network is slow, writes to disks will be delayed and will result in write latency

Solution

  • Check if the increased workload is expected and can be reduced.
    Contact Microsoft to address the Azure network and the Azure disk layers and validate resource contention.

Additional Information

additionalInformation_text

Internal Notes

 
  • The issues with the high write latency were observed on multiple layers of the CVO system, indicating an issue with the backend disk used by the system
  • The workload view shows high WAFL Suspend CP latency, indicating issues while trying to write the data to the disk:

image.png

  • This can be confirmed from the Consistency Point (CP) view, showing Back-to-Back (B2B) CP events:

image (1).png

  • These events had higher P2_FLUSH latency, confirming that the writes to disk on Azure were the root cause:

image (2).png

  • We can also observe NVLOG_FULL Messages for SPINNP_WRITE in WAFL Suspend:

image (3).png

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.