Skip to main content
NetApp Knowledgebase

What is the object fragmentation in fabricpool?

  1. Last updated
  2. Save as PDF
Views:
310
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
sgrid
Last Updated:

Applies to

  • Ontap
  • Fabricpool tiering

Answer

  • FabricPool does not delete blocks from attached object stores.
  • Instead, FabricPool deletes entire objects after a certain percentage of the blocks in the object are no longer referenced by ONTAP.
    • For example, there are 1,024 4KB blocks in a 4MB object tiered to Amazon S3.
  • If a customer/client app makes a delete or a write to a file that has cold blocks in an object, the cold block becomes unreferenced, but it stays in the object.
    • This fragmentation slowly builds up until it crosses the unreclaimed space threshold and we delete the object and fold any existing referenced blocks into a new object.
  • Defragmentation and deletion do not occur until less than 205 4KB blocks (20% of 1,024) are being referenced by ONTAP.
    • When enough (1,024) blocks have zero references, their original 4MB objects are deleted, and a new object is created.

More details available in TR-4598.

Additional Information

  • Object storage is a storage architecture that manages data as objects, as opposed to other storage architectures such as file or block storage. Objects are kept inside a single container (such as a bucket) and are not nested as files inside a directory inside other directories.
  • Evidence of Object fragmentation may include:
    • A disproportionately large number of small GET requests from the ONTAP object store client
    • The small GET requests are contrast to significantly larger workload read requests.
    • Example:
//diag mode
cluster1::*> statistics start -object object_store_client_op -counter get_request_size_histogram -node N2
cluster1::*> statistics show -sample-id sample_21

Object: object_store_client_op
Instance: 1
Start-time: 8/20/2020 02:41:37
End-time: 8/20/2020 02:42:39
Elapsed-time: 354s
Scope: N2

    Counter                                                     Value
    -------------------------------- --------------------------------
    get_request_size_histogram                                      -
                              <=512B                             1029
                               <=8KB                          1942176  
                              <=16KB                            54012 
                              <=32KB                            46181 
                              <=64KB                            43441 
                             <=128KB                            10409 
                             <=256KB                             6166
                             <=512KB                             2137