Skip to main content
NetApp Knowledge Base

CONTAP-589890: Deduplication with scan-old-data never completes and keeps on creating more dedup metadata

  1. Last updated
  2. Save as PDF
Views:
5
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
core
Last Updated:

Issue

  • The issue was only seen on data protection (DP) volumes so far that are are Snapmirror destination volumes.
  • The affected volume is configured with "storage-efficiency-mode" set to "default" and not set to "efficiency"
    Note: As a result, compression is on file/volume level and not on aggregate level (extended-auto-adaptive-compression = false)
    ::> vol efficiency show -vserver MySVM -volume MyVolume -fields storage-efficiency-mode
    vserver volume storage-efficiency-mode extended-auto-adaptive-compression
    -------- --------- ----------------------- ----------------------------------
    MySVM MyVolume default false

  • An efficiency operation, usually a deduplication full scan (where -scan-old-data true was executed in the past, is active since an unexpected long time and shows a "kb scanned" value that is way larger than even the logical size of the backing volume and keeps on increasing over time:
    ::> volume efficiency show -volume gglriesg
    Vserver Volume State Status Progress Policy
    ---------- ---------------- --------- ----------- ------------------ ----------
    MySVM MyVolume Enabled Active 2000071454972 KB Scanned -

    Note: 2000071454972 KB / 1024 / 1024 / 1024 = 1862,7 TB > volume logical-used
  • Unexpected high volume footprint from Temporary Deduplication metadata:
    ::> vol show-footprint MyVolume
    Vserver : MySVM
    Volume : MyVolume
    Feature Used Used%
    -------------------------------- ---------- -----
    Volume Data Footprint 27.07TB 46%
    Volume Guarantee 0B 0%
    Flexible Volume Metadata 204.8GB 0%
    Deduplication Metadata 798.5PB
    Temporary Deduplication 798.5PB <<<<<
    Delayed Frees 378.4GB 1%
    File Operation Metadata 4KB 0%
    Total Metadata Footprint 1013GB 2%
    Total Footprint 28.06TB 47%
    Footprint Data Reduction 12.03TB 20%
    Data Compaction 12.03TB 20%
    Effective Total Footprint 16.03TB 27%

  • The usage will keep on increasing as long as the efficiency operation is active and the operation might never be able to complete, even after months.
  • The "Temporary Deduplication" metadata size is calculated wrong (e.g. in PB rather than TB) and the reporting is addressed via software defect CONTAP-570921.
  • Patching to a fixed release for CONTAP-570921, will report the usage correctly but not stop the issue.

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.