Skip to main content
NetApp Knowledge Base

What is Adaptive QoS and how does it work?

Views:
8,227
Visibility:
Public
Votes:
9
Category:
ontap-9
Specialty:
perf
Last Updated:

Applies to

ONTAP 9.3 or newer

Answer

  • Adaptive QoS uses QoS Throughput Floors and Ceilings (Minimum and Maximum throttle limits) to set individual Volume limits.
    • Floors are used to prevent bully workloads from robbing workloads of their share of resources
      • This keeps workloads from going below a minimum amount of IOPS and/or MB/s
      • Think the inverse of a ceiling
    • Ceilings are used to limit busier workloads to prevent robbing resources from other workloads
  • Adaptive QoS is dynamic based upon volume size, meaning a 10GB volume would have a different Floor and Ceiling than a 10 TB volume.
    • This means the Ceiling is the greater of 1) Expected IOPS, 2) Peak IOPS, or 3) Absolute Minimum IOPS
      • Note: Expected or peak could be determined by if used or allocated space is set, so if calculating, please consider that
    • The Floor is always Expected IOPS unless below the Absolute Minimum IOPS
Terms
Adaptive QoS Dynamic QoS Ceilings and Floors which grow or shrink based upon volume size used or allocated
Throughput Floor (Minimum) A guaranteed throughput measurement (IOPS and/or MB/s) that gives non-floored workloads less priority over floored workloads
Throughput Ceiling (Maximum) A hard limit of how many IOPs a Volume is allocated (regular QoS)
Expected IOPS The Throughput Floor value in IOPS per terabyte (unless specified different)
Absolute minimum IOPS

An IOP Throughput Floor, used when Expected IOPs becomes too low and overrides Expected IOPs or Peak

Example: A 10 GB Volume with a default Adaptive QoS "value" policy group will have a Floor of 75 IOPs, not 1.28 IOPs Expected.

Peak IOPS The value in IOPS per terabyte (unless specified different) is the Ceiling when greater than Expected and Absolute Minimum IOPS
Allocated Space
  • AQoS limits can be set on this
  • Commands which show the calculated value:
    • volume show -fields total
    • df -g not including the .snapshot line
Used Space
  • This is the logical space including snapshot spill and excluding storage-efficiency space savings
  • Command which show the calculated value:
    • volume show -fields logical used
Headroom

Note: The calculated value can be seen with the qos workload show -instance command

Cluster::> qos workload show -instance

          Workload Name: aqos1-wid32444
...
     Maximum Throughput: 1425IOPS

policy-group expected-iops peak-iops absolute-min-iops peak-iops-allocation
extreme 6144IOPS/TB 12288IOPS/TB 1000IOPS used-space
performance 2048IOPS/TB 4096IOPS/TB 500IOPS used-space
value 128IOPS/TB 512IOPS/TB 75IOPS used-space
 
 
How do QoS floors and ceilings decide when to throttle?
  1. IOPS from volumes without a floor will be put into a queue Best Effort
  2. IOPS from volumes with a floor will be put into a deadline queue
  3. IOPS from volumes with floors get priority over volumes without floors in the dblade as long as workload on those volumes is under the floor value.
  4. IOPS from volumes exceeding the floor are treated the same as a volume without a floor.
    • IOPS queue in a delay center called QoS Minimums
    • IOPS in this queue are treated along with those from volumes without a floor in the Best Effort queue
  5. If CPU is above the headroom value:
    1. Below the CPU headroom optimal point, Best Effort IOPS are yielded to deadline IOPS
    2. Above the CPU headroom optimal point on ONTAP 9.6 and below, the deadline IOPS may be lower but will still get priority over Best Effort IOPS
    3. Above the CPU headroom optimal point on ONTAP 9.7 and above, deadline IOPS will get the same values as below the optimal point and Best Effort IOPS are throttled heavier
  6. IOPS from volumes hitting the Throughput Ceiling will be hard throttled at that value.

Additional Information

 

  • Here is some additional examples of space usage and throttling levels for comparison:
    • Expected IOPS = 200/TB
    • Peak IOPS = 1000/TB
    • Absolute Min IOPS = 75
    • Expected IOPs = allocated space
    • Peak IOPs = used space
  • The minimum and maximum IOPS will be calculated as follows for used-space allocation:

Volume Size

Data Stored

QoS Min IOPS (SSD Aggregate only)

QoS Max IOPS

1 GB

0GB

75 (Absolute Minimum)

75 (Absolute Minimum)

1 TB

0 TB

200 (Expected)

200 (Expected)

1 TB

.1 TB

200 (Expected)

200

1 TB

.2 TB

75 (Absolute Minimum)

200 (Peak and Expected)

1 TB

.4 TB

80 (Expected)

400 (Peak)

1 TB

.5 TB

100

500

2 TB

1.5 TB

300

1500

2 TB

1.75 TB

350

1750

 

 

Scan to view the article on your device