Why are my FlexGroup constituents not balanced?

Applies to

• ONTAP 9
• FlexGroup

Answer

• In an optimally balanced FlexGroup volume, all constituents have roughly the same amount of data and load, and the volume can maintain that state while using a high frequency of local placement for best performance.
  • A workload with a good balance of folders and similarly sized files would be able to maintain local parent folder placement while also keeping a relatively even balance of capacity.
• A less optimal FlexGroup volume might have some constituents that hold more or less data than their peers, or that are receiving much more or much less traffic.
  • Workloads that have only a few folders with many files per folder, or workloads with highly variant file sizes can experience data usage imbalances in a FlexGroup.
• Capacity balance, however, is not the most important function of a FlexGroup volume.
  • Instead, a FlexGroup volume functions best when there is a mix of local placement for performance along with capacity and inode count balance.
  • We don’t want to sacrifice performance for the sake of perfectly balanced capacities across member volumes.
• ONTAP constantly monitors the ongoing state of the member volumes and adjusts placement decisions based on the current state of the FlexGroup volume.
  • Prior ONTAP 9.8, if one member volume is a little out of balance from the others, then it’s likely that no adjustments will be made to ingest.
  • But if that member volume starts to approach 90% capacity, or has a capacity discrepancy that exceeds 512GB, then ONTAP makes more aggressive placement choices for new data to correct the imbalance.
  • This adjustment means more remote file placement to other member volumes, which can have a negative (but potentially unnoticeable―roughly 5% to 10%) effect on FlexGroup volume performance.
• ONTAP 9.8 changed the 90% threshold for data placement urgency (where ONTAP forces more data to other less full member volumes much more frequently when a member volume hits 90% capacity) to a higher value (99%) so that FlexGroup member volumes that are growing in size won’t adjust their data ingest rate by very much until they reach 99% capacity, eliminating the negative impact of remote file placement in the earlier versions.
• After a FlexVol to FlexGroup conversion, followed by FlexGroup expansion, the used space on the initial constituent may be significantly different to the constituents automatically created by ONTAP following the expansion. Expanding the FlexGroup will not balance the existing data.
• ONTAP 9.10.1 introduces a disruptive process to rebalance FlexGroups; volume rebalance file-move
• Beginning with ONTAP 9.12.1, rebalance FlexGroup volumes by non-disruptively moving files from one constituent in a FlexGroup to another constituent. Additional information can be found in Rebalance FlexGroup volumes

Additional Information

=================================================

Best Practice 2: Stop worrying about capacity imbalances.

=================================================

• A FlexGroup with a capacity imbalance across member volumes is not in itself a problem and should not be treated as such.
• Instead, look at capacity imbalances as a potential cause if a FlexGroup is not performing as expected or if the capacity imbalances are so extreme that you’re running out of space on your cluster.
• Be sure to engage NetApp support if you feel that your FlexGroup volume capacity imbalance is the source of a performance issue.
• TR-4571: NetApp ONTAP FlexGroup Volumes Best Practices and Implementation Guide
• TR-4557: NetApp ONTAP FlexGroup Volumes A Technical Overview
• High CPU utilization due to FlexGroup constituents imbalance