Large batches of small file copies are slower after migration from on-prem AFF to Azure CVO
- Views:
- 80
- Visibility:
- Public
- Votes:
- 0
- Category:
- cloud-volumes-ontap-cvo
- Specialty:
- microsoft_azure
- Last Updated:
Applies to
- CLOUD VOLUMES ONTAP
- ONTAP 9
Issue
- After migrating from on-premises ONTAP (AFF) to CVO in Azure, workloads that generate or write large batches of small files (e.g., statement generation jobs involving 15,000–50,000 files) experience significantly slower performance.
- Previously, such jobs completed in 1–4 hours; after migration, the same jobs take 8 or more hours to complete.
Cause
-
CVO NVRAM/NVLOG Latency:
-
Cloud Volumes ONTAP uses a virtual NVRAM (NVLOG) disk backed by Azure Premium SSD.
-
This emulation is inherently slower than the hardware NVRAM used in AFF/FAS systems, resulting in higher write latencies, especially with high volumes of small, sequential write operations.
-
-
Low Application Concurrency:
-
The application writing the files uses very low concurrency (often single-threaded), causing operations to be serialized.
-
Any increase in storage or network latency is multiplied across thousands of sequential operations, greatly extending total job run time.
-
Solution
-
Deploy a new CVO instance using Premium SSD v2 and UltraSSD for NVRAM:
- Premium SSD v2 disks and UltraSSD-backed NVRAM provide significantly lower latency and higher throughput compared to previous disk types.
- Plan and execute a migration to the new CVO instance with these disk types.
-
Enable High Write Speed Mode on CVO:
-
High Write Speed mode bypasses some write buffering and can reduce write latency for workloads similar to on-premises performance.
-