Skip to main content
NetApp Knowledge Base

What is Adaptive QoS and how does it work?

Views:
15,131
Visibility:
Public
Votes:
13
Category:
ontap-9
Specialty:
perf
Last Updated:

Applies to

  • ONTAP 9.3 or newer
  • Adaptive QoS (AQoS)
  • Active IQ Unified Manager (AIQUM)
  • NetApp Service Level Manager (NSLM)

Answer

  • Adaptive QoS uses QoS Throughput Floors and Ceilings (Minimum and Maximum throttle limits) to set individual Volume limits.
    • Floors are used to prevent bully workloads from robbing workloads of their share of resources
      • This keeps workloads from going below a minimum amount of IOPS and/or MB/s
      • Think of AQoS as the inverse of a ceiling or traditional QoS policy: it keeps other workloads from taking resources from the workload the policy is applied to
    • Ceilings are used to limit busier workloads to prevent robbing resources from other workloads
  • Adaptive QoS is dynamic based upon volume size, meaning a 10GB volume would have a different Floor and Ceiling than a 10 TB volume.
    • This means the Ceiling is the greater of 1) Expected IOPS, 2) Peak IOPS, or 3) Absolute Minimum IOPS
      • Note: Expected or peak could be determined by if used or allocated space is set, so if calculating, please consider that
    • The Floor is always Expected IOPS unless below the Absolute Minimum IOPS
  • As with regular QoS, AQoS is a cluster wide process as I/O may hit any LIF on any node in the Cluster
Terms
Adaptive QoS Dynamic QoS Ceilings and Floors which grow or shrink based upon volume size used or allocated
Throughput Floor (Minimum) A guaranteed throughput measurement (IOPS and/or MB/s) that gives non-floored workloads less priority over floored workloads
Throughput Ceiling (Maximum) A hard limit of how many IOPs a Volume is allocated (regular QoS)
Expected IOPS The Throughput Floor value in IOPS per terabyte (unless specified different)
Absolute minimum IOPS

An IOP Throughput Floor, used when Expected IOPs becomes too low and overrides Expected IOPs or Peak

Example: A 10 GB Volume with a default Adaptive QoS "value" policy group will have a Floor of 75 IOPs, not 1.28 IOPs Expected.

Peak IOPS
  • The value in IOPS per terabyte (unless specified different) is the Ceiling when greater than Expected and Absolute Minimum IOPS.
  • Example:
    • A 1.23 TB volume with used space of 566.66 GB exists with a default Extreme policy.
    • Expected and Peak IOPS are the same at 7,560 IOPS because:
      • Used space * 12288 (Peak IOPS * used) = 6,800 IOPS
      • Allocated space * 6144 (Expected IOPS * allocated) = 7,560 IOPS
      • Since 6,800 Peak IOPS < 7,560 Expected IOPS, Peak IOPS will round up to the 7,560 IOPS.
Allocated Space
  • AQoS limits can be set on this
  • Commands which show the calculated value:
    • volume show -fields total
    • df -g not including the .snapshot line
Used Space
  • This is the logical space including snapshot spill and excluding storage-efficiency space savings
  • Command which show the calculated value:
    • volume show -fields logical used
Headroom

Note: The calculated value can be seen with the qos workload show -instance command

Cluster::> qos workload show -instance

          Workload Name: aqos1-wid32444
...
     Maximum Throughput: 1425IOPS

policy-group expected-iops peak-iops absolute-min-iops expected-iops-allocation peak-iops-allocation
extreme 6144IOPS/TB 12288IOPS/TB 1000IOPS allocated-space used-space
performance 2048IOPS/TB 4096IOPS/TB 500IOPS allocated-space used-space
value 128IOPS/TB 512IOPS/TB 75IOPS allocated-space used-space
 
 
How do QoS floors and ceilings decide when to throttle?
  1. IOPS from volumes without a floor will be put into a queue Best Effort
  2. IOPS from volumes with a floor will be put into a deadline queue
  3. IOPS from volumes with floors get priority over volumes without floors in the dblade as long as workload on those volumes is under the floor value.
  4. IOPS from volumes exceeding the floor are treated the same as a volume without a floor.
    • IOPS queue in a delay center called QoS Minimums
    • IOPS in this queue are treated along with those from volumes without a floor in the Best Effort queue
  5. If CPU is above the headroom value:
    1. Below the CPU headroom optimal point, Best Effort IOPS are yielded to deadline IOPS
    2. Above the CPU headroom optimal point on ONTAP 9.6 and below, the deadline IOPS may be lower but will still get priority over Best Effort IOPS
    3. Above the CPU headroom optimal point on ONTAP 9.7 and above, deadline IOPS will get the same values as below the optimal point and Best Effort IOPS are throttled heavier
  6. IOPS from volumes hitting the Throughput Ceiling will be hard throttled at that value.

Additional Information

  • Here is some additional examples of space usage and throttling levels for comparison:
    • Expected IOPS = 128/TB
    • Peak IOPS = 512/TB
    • Absolute Min IOPS = 75
    • Expected IOPs = allocated space
    • Peak IOPs = used space

Volume Size

Data Stored

QoS Min IOPS (SSD Aggregate only)

QoS Max IOPS

1 GB

0GB

75 (Absolute Minimum)

75 (Absolute Minimum)

1 TB

0 TB

128 (Expected)

128 (Expected)

1 TB

.1 TB

128 (Expected)

128 (Expected)

1 TB

.2 TB

128 (Expected)

128 (Expected)

1 TB

.3 TB

128 (Expected)

154 (Peak)

1 TB

.5 TB

128 (Expected)

256 (Peak)

1 TB

1 TB

128 (Expected)

512 (Peak)

2 TB

2 TB

256 (Expected)

1024 (Peak)

 

 

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.